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I 3  ABSTRAC  T 


This  paper  analyzes  ten  methods  that  have  been  used  or  proposed  for  planning  the  allo¬ 
cation  of  resources  among  projects  within  the  Exploratory  Development  category  of  the  Defense 
Research,  Development,  Test,  and  Evaluation  Program.  Each  method  is  described  in  terms  of  a 
general  framework  of  planning  methods  and  of  the  factors  that  influence  the  allocation  of  de¬ 
velopment  resources.  A  comparative  analysis  is  made  of  the  relative  strengths  and  weaknesses 
of  these  methods.  ' 

The  methods  generally  fall  into  two  groups.  Those  in  the  more  quantitative  group  try  to 
express  all  important  factors  numerically,  derive  a  single  numerical  measure  of  merit  for  pro¬ 
posed  projects,  and  calculate  precisely  an  optimum  allocation  of  resources  among  the  projects. 
Those  methods  in  the  less  quantitative  group  do  not  try  to  measure  numerically  some  important 
factors  and  leave  the  derivation  of  the  allocation  of  resources  among  the  various  projects  to 
the  judgment  of  the  responsible  decision  maker. 

The  more  quantitative  methods  are  uniformly  more  complex  and  difficult  to  apply.  Such 
methods  also  generally  fail  to  treat  all  of  the  important  factors  in  the  allocation  problem 
accurately  and  adequately.  For  example,  the  risks  involved  in  planning  the  technical,  timing, 
and  cost  results  of  individual  and  groups  of  efforts  are  virtually  ignored.  Also,  individual 
projects  are  treated  as  though  they  are  quite  independent  of  each  other.  Consequently,  none 
of  these  methods  can  be  recommended  for  use  in  their  current  form. 

The  less  quantitative  methods  are  primarily  frameworks  for  recording  and  transmitting 
information  that  is  important  to  the  formulation  of  the  development  program.  Because  they 
do  not  attempt  to  devise  a  single  optimum  allocation  of  development  resources,  they  are  less 
likely  to  mislead  program  management  than  the  more  quantitative  methods.  Consequently,  man¬ 
agers  may  find  some  one  of  these  to  be  a  convenient  framework  for  organizing  the  information 
that  they  want  readily  available  for  deciding  the  allocation  of  development  resources. 
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ABSTRACT 


This  paper  analyzes  ten  methods  that  have  been  used  or  proposed 
for  planning  the  allocation  of  resources  among  projects  within  the 
Exploratory  Development  category  of  the  Defense  Research,  Development, 
Test,  and  Evaluation  Program.  Each  method  is  described  in  terms  of 
a  general  framework  of  planning  methods  and  of  the  factors  that  in¬ 
fluence  the  allocation  of  development  resources.  A  comparative  analysis 
is  made  of  the  relative  strengths  and  weaknesses  of  these  methods. 

The  methods  generally  fall  into  two  groups.  Those  in  the  more 
quantitative  group  try  to  express  all  important  factors  numerically, 
derive  a  single  numerical  measure  of  merit  for  proposed  projects,  and 
calculate  precisely  an  optimum  allocation  of  resources  among  the  proj¬ 
ects.  Those  methods  in  the  less  quantitative  group  do  not  try  to 
measure  numerically  some  important  factors  and  leave  the  derivation  of 
the  allocation  of  resources  among  the  various  projects  to  the  judgment 
of  the  responsible  decision  maker. 

The  more  quantitative  methods  are  uniformly  more  complex  and  dif¬ 
ficult  to  apply.  Such  methods  also  generally  fail  to  treat  all  of  the 
important  factors  in  the  allocation  problem  accurately  and  adequately. 
For  example,  the  risks  involved  in  planning  the  technical,  timing,  and 
cost  results  of  individual  and  groups  of  efforts  are  virtually  ignored. 
Also,  individual  projects  are  treated  as  though  they  are  quite  inde¬ 
pendent  of  each  other.  Consequently,  none  of  these  methods  can  be 
recommended  for  use  in  their  current  form. 

The  less  quantitative  methods  are  primarily  frameworks  for  re¬ 
cording  and  transmitting  information  that  is  important  to  the  formu¬ 
lation  of  the  development  program.  Because  they  do  not  attempt  to 
devise  a  single  optimum  allocation  of  development  resources ,  they  are 
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less  likely  to  mislead  program  management  than  the  more  quantitative 
methods.  Consequently,  managers  may  find  some  one  of  these  to  be  a 
convenient  framework  for  organizing  the  information  that  they  want 
readily  available  for  deciding  the  allocation  of  development  resources. 


vi 


CONTENTS 


I.  INTRODUCTION,  SUMMARY  AND  CONCLUSIONS  1 

A.  Purpose  1 

B.  Procedure  1 

C.  Summary  Description  of  Methods  2 

D.  Conclusions  2 

E.  Recommendations  7 

II.  GENERAL  MODEL  11 

A.  Need  for  General  Model  11 

B.  Structure  of  the  General  Model  11 

III.  DESCRIPTIONS  OF  QUANTITATIVE  METHODS  17 

A.  Industrial  Analog  17 

B.  TORQUE  22 

C.  Naval  Ordnance  Laboratory  Method  33 

D.  Air  Force  Flight  Dynamics  Laboratory  Method  41 

E.  Cornell  Aeronautical  Laboratory  Method  49 

F.  Hercules  Method  56 

G.  Army  Missile  Plan  60 

H.  Air  Force  DOL  Plan  65 

I.  Army  Research  Plan  68 

J.  Another  Service  Method  75 

IV.  CRITIQUE  OF  QUANTITATIVE  METHODS  79 

A.  Primary  Objective  of  a  Quantitative  Method  81 

B.  Control  Variables  88 

C.  Factors  and  Their  Relationships  92 

D.  Constraints  140 

E.  Decision  Algorithm  146 

F.  Findings  on  Each  Method  156 

G.  General  Discussion  167 

Appendix — TORQUE  Budget  Allocation  Algorithm  175 

References  179 


vii 


ABBREVIATIONS 


ACORD  Advanced  concepts  for  ordnance 

ADO  Advanced  development  objective 

AFSC  Air  Force  Systems  Command 

AFRPL  Air  Force  Rocket  Propulsion  Laboratory 

AMC  Army  Materiel  Command 

BRAILLE  Balanced  resource  allocation  information  for  logical 

lucid  evaluation 

CAL  Cornell  Aeronautical  Laboratory 

CDC  Combat  Development  Command 

CDOG  Combat  development  objectives  guide 

CDP  Contract  definition  phase 

CL  Confidence  level 

DA  Department  of  the  Army 

DPQMDO  Draft  proposed  qualitative  materiel  development  objective 

DOL  Directorate  of  Laboratories  (Air  Force) 

EDG  Exploratory  development  goal  (Navy) 

FDL  Flight  Dynamics  Laboratory  (Air  Force) 

I.D.  Interdisciplinary  (Team) 

IR&D  Independent  research  and  development 

LAW  Light  antitank  weapon 

MACRO  R&D  Methodology  for  allocating  corporate  resources  to  objec¬ 
tives  for  R&D 


MARS 


Multiple  artillery  rocket  system 


NOL 

NTF 

OCD 

PQMDO 

PQMR 

QMDO 

QMR 

QUEST 

SFC 

SHORAD 

SMEADO 

TORQUE 

TPO 

TRAADS 


Naval  Ordnance  Laboratory 
Navy  Technological  Forecast 

Operational  capability  objective 

Proposed  qualitative  materiel  development  objective 
Proposed  qualitative  materiel  requirement 

Qualitative  materiel  development  objective  (Army) 
Qualitative  materiel  requirement 

Qualitative  utility  estimates  for  science  and  technology 

Specific  fuel  consumption 
Short-range  air  defense 

Selected  major  exploratory  and  advanced  development 
objectives 

Technology  or  research  quantitative  utility  evaluation 

Technical  planning  objective 

Technical  Review,  Army  Air  Defense  Systems 


x 


I.  INTRODUCTION,  SUMMARY  AND  CONCLUSIONS 


A.  PURPOSE 

The  purpose  of  this  Paper  is  to  examine  in  detail  the  critical 
features  of  ten  quantitative  methods  that  may  be  used  to  allocate 
resources  within  the  Exploratory  Development  category  of  the  DoD 
Research,  Development,  Test,  and  Evaluation  Program.  It  supports 
IDA  Paper  P-652,  "Quantitative  Methods  for  the  Allocation  of  DoD 
Exploratory  Development  Resources"  (Ref.  1).  The  ten  quantitative 
methods  examined  are  listed  below: 

1.  Industrial  Analog 

2 .  TORQUE 

3.  Naval  Ordnance  Laboratory  Method 

4.  Air  Force  Flight  Dynamics  Laboratory  Method 

5.  Cornell  Aeronautics  Laboratory  Method 

6.  Hercules  Corporation  Method 

7.  Army  Missiles  Plan 

8.  Air  Force  Directorate  of  Laboratories  Plan 

9.  Army  Research  Plan 

10.  Another  Service  Method 

No  effort  is  made  to  review  all  the  literature  of  the  field  in 
this  paper;  extensive  bibliographies  have  been  compiled  by  Baker  and 
Pound  (Ref.  2),  and  Cetron,  Martino,  and  Roepke  (Ref.  3). 

B.  PROCEDURE 

To  carry  out  the  review,  a  general  frame  of  reference  is  de¬ 
veloped  in  the  next  chapter.  This  provides  a  convenient  structure 
for  comparing  the  various  features  of  the  quantitative  methods .  In 
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Chapter  III ,  each  method  is  described  within  this  framework  in  order 
to  make  the  similarities  and  differences  of  the  various  methods  more 
evident.  In  Chapter  IV,  the  methods  are  analyzed  together  according 
to  each  element  of  the  general  framework  to  highlight  their  relative 
strengths  and  weaknesses. 

This  analysis  uses,  in  part,  information  that  was  developed  from 
a  number  of  visits  to  Government  and  industry  laboratories  and  from 
interviews  of  professionals  and  managers  engaged  in  this  type  of  de¬ 
velopmental  work. 

C.  SUMMARY  DESCRIPTION  OF  METHODS 

Table  1  summarizes  the  structures  and  main  features  of  the  ten 
methods  that  are  reviewed  in  this  study.  Each  is  described  according 
to  how  it  deals  with  such  factors  as  value  measures,  operational  re¬ 
quirements,  technologies,  costs,  and  risks. 

D.  CONCLUSIONS 

The  ten  methods  fall  into  two  groups:  a  more  quantitative  group 
and  a  less  quantitative  group.  The  more  quantitative  methods  (In¬ 
dustrial  Analog,  TORQUE,  Naval  Ordnance  Laboratory,  Flight  Dynamics 
Laboratory,  Cornell  Aeronautical  Laboratory,  Hercules,  and  Another 
Service)  structure  the  allocation  problem  and  express  its  factors  al¬ 
most  exclusively  in  mathematical  terms.  They  generally  devise  a  single 
numerical  measure  of  merit  for  each  proposed  development  project  and 
use  this  measure  of  merit  in  formal  calculations  of  the  precise  allo¬ 
cation  of  resources  that  should  be  made  to  each  project. 

The  less  quantitative  methods  (Army  Missile  Plan,  Air  Force  Direc¬ 
torate  of  Laboratories  Plan,  and  Army  Research  Plan)  do  not  try  to 
express  mathematically  some  important  factors  in  the  allocation  problem. 
They  place  much  greater  reliance  upon  managerial  judgment  to  take 
such  factors  into  account.  These  methods  recommend  an  allocation  of 
resources  among  proposed  development  efforts  but  without  the  numerical 
precision  of  the  more  quantitative  methods  and  without  contending  that 
the  recommendation  is  the  single,  most  desirable  allocation. 
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TABLE  1.  SUMMARY  DESCRIPTION  OF  METHODS  FOR 
ALLOCATING  DEVELOPMENT  RESOURCES 
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10. 

The  conclusions  are  arranged  according  to  whether  they  apply 
primarily  to  one  of  these  groups  or  whether  they  apply  to  the  methods 
in^  general. 

1.  More  Quantitative  Methods 

Most  of  the  more  quantitative  methods  treat  individual  develop¬ 
ment  efforts  ,  weapon  systems ,  and  operational  requirements  as  though 
their  completion  is  independent  of  other  efforts,  systems,  and  re¬ 
quirements,  respectively.  This  independence  has  not  been  demonstrated 
and  probably  does  not  hold  in  many  significant  cases. 

The  procedure  used  in  these  methods  to  impute  military  value  to 
development  efforts,  weapon  systems,  and  operational  requirements  re¬ 
quires  that  the  efforts  ,  systems ,  and  requirements  be  independent  of 
each  other.  To  the  extent  that  the  independence  does  not  hold,  the 
values  assigned  to  these  factors  may  be  highly  inaccurate. 

The  linkage  of  the  technological  efforts  to  weapon  systems  and 
operational  requirements  is  formulated  in  many  of  these  methods  in 
such  a  way  that  the  technological  efforts  can  determine  the  emphasis 
that  will  be  placed  on  the  development  of  various  weapon  systems  and 
on  the  fulfillment  of  the  various  operational  requirements.  This  re¬ 
verses  the  procedural  order  by  which  the  development  program  should 
be  derived . 

These  methods  are  not  specific  about  the  scope  of  the  resources 
that  should  be  taken  into  account  in  the  allocation  problem.  For 
example,  they  give  no  guidance  on  the  treatment  of  such  costs  as  the 
rents  that  should  be  imputed. to  the  use  of  Government -owned  real  es¬ 
tate  and  the  payments  for  resources  that  are  financed  from  appropria¬ 
tions  other  than  the  Research,  Development,  Test,  and  Evaluation 
appropriation.  The  methods  also  do  not  propose  how  to  treat  the  time 
distribution  of  funding  requirements  in  the  allocation  problem. 

These  methods  do  not  formulate  adequately  a  framework  for  con¬ 
sidering  how  actual  technical,  cost,  and  timing  outcomes  of  a  develop¬ 
ment  program  may  jointly  deviate  from  the  point  estimates  made  of  these 
factors  at  the  time  the  allocation  decision  is  reached. 
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The  workings  of  these  methods  are  probably  comprehensible  to 
few  of  the  participants  in  the  allocation  process  outside  the  special¬ 
ists  who  formulate  the  methods  because  of  the  complexity  of  (1)  the 
formal  decision  algorithms  and  (2)  the  treatment  given  such  factors 
as  risk  and  the  relationships  among  technologies  in  the  composition 
of  a  weapon  system. 

2.  Less  Quantitative  Methods 

The  less  quantitative  methods  attempt  primarily  (1)  to  record 
information  on  some  relevant  considerations  that  must  be  made  to 
determine  the  allocation  of  the  development  budget  and  (2)  to  trans¬ 
mit  this  information  visibly  to  all  levels  of  management. 

These  methods  generally  have  more  narrowly  defined  objectives 
than  the  more  quantitative  methods ,  focusing  tightly  on  deriving  a 
program  of  technological  advances  that  would  support  a  very  specific 
set  of  projected  weapon  systems.  Consequently,  the  chosen  develop¬ 
ment  projects  should  be  both  consistent  with  each  other  and  with  the 
development  of  some  set  of  weapons. 

Being  tightly  focused,  these  methods  are  not  likely  to  consider 
a  set  of  development  efforts  as  broad  as  that  brought  to  the  surface 
in  the  more  quantitative  methods. 

These  methods  rely  heavily  upon  top  management’s  judgment  to 
consider,  without  guidance,  factors  such  as  (1)  the  technical,  timing, 
and  cost  risks  involved  in  a  development  effort  and  (2)  the  importance 
to  a  weapon  system  of  a  technological  advance  produced  by  a  development 
effort.  This  leaves  implicit  much  of  the  rationale  for  the  final  al¬ 
location  so  that  it  is  difficult  to  review  the  rationale  and  to  impose 
the  allocation  as  an  objective  standard  on  other  management  and  pro¬ 
fessional  personnel  for  implementation. 

These  less  quantitative  methods  may  not  choose  the  development 
projects  best  supporting  higher  order  Defense  goals  because  of  the 
limited  number  of  options  they  consider  and  the  lack  of  precision  in 
their  treatment  of  important  factors . 
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3 .  Both  Groups  of  Methods 

All  of  the  method  reviewed  analyze  the  allocation  of  develop¬ 
ment  resources  that  should  be  made  in  some  future  period.  None  analyze 
past  resource  allocations  and  project  outcomes  to  develop  a  data  base 
for  applications  of  the  method. 

None  of  the  methods  measure  the  dispersion  of  estimates  that 
might  be  made  by  a  number  of  technologists  and  managers  for  projected 
factors  such  as  costs,  timing,  technical  advance,  the  importance  of 
a  technical  advance,  and  the  importance  of  a  weapon  system. 

None  of  the  methods  make  provision  for  tests  of  internal  con¬ 
sistency.  No  tests  appear  to  have  been  performed  or  proposed  to  show 
the  differences  in  allocations  that  might  result  (1)  from  the  appli¬ 
cation  of  the  method  by  various  decision  groups  to  the  same  conditions 
or  (2)  from  the  application  of  the  method  by  the  same  group  to  the 
same  conditions  at  various  times. 

None  of  the  methods  have  proposed  a  measure  and  procedure  to 
test  (1)  how  closely  the  resource  allocation  it  proposes  is  actually 
implemented  and  (2)  if  its  prescribed  allocations  are  followed,  how 
much  better  its  development  programs  are  relative  to  (a)  programs 
prescribed  by  other  methods  or  (b)  programs  devised  without  any 
formal  allocation  method. 

E.  RECOMMENDATIONS 

Because  of  the  shortcomings  listed  above  ,  none  of  the  more  quan¬ 
titative  methods  should  be  applied,  in  their  current  formulations,  to 
determining  the  allocation  of  resources  within  Defense  Exploratory 
Development. 

Development  of  more  quantitative  methods  for  allocating  resources 
within  Exploratory  Development  should  continue,  however,  because  their 
rigorous  structure  should  provide  (1)  a  useful  frame  of  reference  for 
organizing  information  and  regular  liaison  among  systems  specialists, 
technologists,  and  research  managers  and  (2)  a  systematic  procedure 
for  searching  out  a  development  program  that  will  best  fulfill  Defense 
goals . 
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•  An  investigation  should  be  made  of  the  delegation  of 
goals  by  large ,  diversified  companies  to  their  research 
organizations  for  resemblances  to  the  decentralized 
Defense  Research  and  Development  problem. 

•  More  intensive  study  should  be  given  to  adapting  to 
more  quantitative  methods  the  objective  of  funding  tech¬ 
nologies  to  support  the  set  of  future  weapon  systems 
that  would  maximize  possible  cost  savings  in  performance 
of  a  fixed  mission.  The  HINDSIGHT  (Ref.  4)  study  of  the 
cost  savings  generated  by  the  C-141  and  the  AN/SPS-48 
would  provide  a  useful  starting  point  for  this  work. 

•  Continuing  study  will  be  necessary  to  improve  the  quan¬ 
titative  expressions  for  (1)  such  important  factors  as 
cost,  timing,  technical  advance,  and  weapon  performance; 
(2)  possible  variances  between  predicted  and  actual 
values  for  these  factors;  and  (3)  the  relationships  of 
the  technological  components  of  a  system  and  its  per¬ 
formance. 

While  more  quantitative  methods  are  being  developed  ,  RSD  managers 
should  consider  adopting  the  framework  of  a  less  quantitative  method 
for  organizing  the  information  they  should  have  readily  available  to 
determine  the  allocation  of  development  resources. 

If  a  less  quantitative  method  such  as  those  reviewed  is  adopted 
as  an  interim  procedure,  steps  should  be  taken  to  incorporate  into 
them  more  explicit  and  precise  expressions  for  (1)  the  technical, 
timing,  and  cost  risks  in  a  development  effort;  (2)  the  relationship 
of  a  technical  advance  to  a  projected  weapon  system;  and  (3)  the  re¬ 
lationship  of  the  various  weapons  to  the  fulfillment  of  the  various 
operational  requirements.  Improvements  must  also  be  made  in  the  cost 
concepts  that  should  be  used  in  the  allocation  procedure. 

To  demonstrate  the  reliability  and  benefits  of  any  method  for 
allocating  development  resources,  further  work  on  these  methods  should 
include  the  devising  of  measures  for  three  kinds  of  tests: 
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•  The  dispersion  of  estimates  by  a  number  of  technologists 
and  managers  of  the  expected  values  for  important  factors 
within  the  prospective  development  program  such  as  timing, 
costs,  technological  advance,  and  operational  requirement 
importance ; 

•  The  differences  in  the  allocations  that  could  result 
from  (1)  the  application  of  the  same  method  by  a  constant 
group  of  decision  makers  to  the  same  conditions  at  differ¬ 
ent  times,  (2)  the  application  of  various  methods  by  a 
constant  group  of  decision  makers  to  the  same  conditions, 
and  (3)  the  application  of  the  same  method  by  various 
groups  of  decision  makers  to  the  same  conditions ; 

•  The  extent  to  which  the  development  program  generated 
by  a  method  is  better  than  the  program  generated  by  any 
other  method  or  by  informal  decision  rules. 

To  facilitate  the  completion  of  such  tests  and  the  compilation  of 
data  that  would  be  useful  in  the  applications  of  the  methods ,  further 
work  should  also  include  provision  for  continuous  collection  of  in¬ 
formation  on  actual  resource  expenditures,  timing,  technological  ad¬ 
vance,  and  applications  of  results  of  development  efforts,  in  the 
same  organizational  units  and  formats  as  those  used  in  the  allocation 
method . 
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II.  GENERAL  MODEL 


A.  NEED  FOR  GENERAL  MODEL 

Initial  study  of  the  ten  methods  for  allocating  development  re¬ 
sources  gives  a  strong  impression  of  diversity  among  them.  Wide  dif¬ 
ferences  exist  in  their  structure,  in  the  general  approaches  they  take 
to  the  allocation  problem,  and  in  the  factors  they  take  into  account. 

Review  of  these  quantitative  methods,  therefore,  requires  a  con¬ 
ceptually  convenient,  common  framework  of  reference  for  organizing  the 
basic  features  of  the  various  methods.  For  present  purposes,  this 
framework  is  called  a  General  Model.  Such  a  General  Model  has  at 
least  two  advantages:  first,  it  helps  resolve  the  differences  in  ex¬ 
position  among  the  various  methods  into  parallel  components;  second, 
it  lends  a  similar  perspective  to  a  review  of  the  several  methods. 

In  turn,  use  of  the  General  Model  helps  to  focus  attention  on 
a  number  of  relevant  questions  about  quantitative  methods.  For  ex¬ 
ample:  What  does  the  method  attempt  to  accomplish?  Is  that  beneficial 
or  useful?  What  factors  does  the  method  take  into  account?  How  does 
the  method  proceed  to  accomplish  its  objective?  Are  the  procedural 
steps  valid?  Are  the  factors  considered  correctly?  Is  the  resulting 
allocation  consistent  with  the  objective  of  the  method? 

B.  STRUCTURE  OF  THE  GENERAL  MODEL. 

There  is  no  unique  General  Model  into  which  the  features  of  quan¬ 
titative  methods  for  allocating  resources  must  or  should  be  organized. 

A  quantitative  method  may  be  broken  down  in  different  ways ,  depending 
upon  the  purpose  of  the  analysis.  The  particular  General  Model  that 
is  reported  below  and  illustrated  in  Fig.  1  has  been  useful  for  both 
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general  and  technical  analyses  of  the  methods  chosen  for  review  in 
this  paper. 


FIGURE  1.  General  Model 


1.  Primary  Objective  of  the  Method 

All  of  the  quantitative  methods  that  were  reviewed  are  directed 
at  allocating  development  resources  or  choosing  development  programs; 
however,  the  different  methods  do  frequently  diverge  in  what  they 
conceive  the  development  program  should  accomplish.  The  purpose  of 
the  development  program  determined  by  the  quantitative  allocation 
method  is  treated  in  this  paper  as  the  primary  objective  of  the 
method. 
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Development  programs  may  have  more  than  one  objective,  or  hier¬ 
archies  of  primary  and  secondary  objectives.  Although  the  possibility 
of  such  hierarchies  has  been  considered  throughout  the  reviews,  most 
cases  can  be  resolved  into  a  structure  with  a  dominant  objective, 
subject  in  some  cases  to  secondary  objectives. 

2 .  Control  Variables 

Inasmuch  as  a  quantitative  method  for  allocating  development 
resources  is  a  tool  for  helping  management  at  some  level  devise  a 
development  program,  it  must  contain  a  concept  of  the  factors  over 
which  management  wants  to  exercise  the  power  of  decision.  These 
factors  are  termed  the  control  variables  in  the  General  Model.  Manage¬ 
ment  determines  the  magnitudes  or  go/no-go  conditions  for  the  control 
variables.  Of  course,  other  factors  may  be  consciously  affected  by 
the  choices  made  for  the  control  variables  but,  for  the  purposes  of 
examining  the  quantitative  methods  ,  this  influence  is  regarded  as  an 
indirect  one. 

Distinguishing  the  control  and  other  variables  in  a  quantitative 
method  is  relatively  important  for  review  purposes  because  it  must 
treat  consistently  what  management  controls,  determines  indirectly, 
and  accepts  as  given. 

3 .  Factors  and  Their  Relationships 

The  factors  incorporated  into  a  quantitative  method  are  the 
elements  that  the  formulator  of  the  method  judges  to  have  important 
effects  upon  the  real  development  process  and,  therefore,  should 
influence  the  allocation  procedure  of  the  method.  Functional  relation¬ 
ships  among  the  factors  are  the  formulator rs  mathematical  or  other 
representations  of  how  factors  influence  one  another,  how  control 
variables  (a  special  class  of  factors)  influence  or  are  influenced  by 
other  factors,  and  how  all  the  factors  affect  the  measure  of  the 
primary  objective  of  the  development  program. 

Taken  together,  the  factors  and  their  relationships  can  be  con¬ 
sidered  to  be  the  quantitative  method’s  characterization  of  the 
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development  process,  describing  the  range  of  development  work  and 
allocations  that  are  feasible  within  the  terms  of  the  method.  The 
quality  of  the  characterization  directly  affects  the  quality  of  the 
allocation  that  will  be  made  by  the  method,  inaccuracies  in  it  possibly 
resulting  in  impossible  or  unbeneficial  allocation  schemes. 

Specific  factors  that  are  taken  into  account  can  generally  be  com¬ 
bined  according  to  the  broader  classes  of  features  and  activities  in 
the  development  process  that  are  described  below. 

a.  Value  Measures.  Quantitative  methods  generally  assign  to  the 
technological  developments  that  are  expected  from  a  particular  alloca¬ 
tion  of  resources  some  overall  measure  of  value  for  the  agency  under¬ 
taking  the  development  program.  These  measures  usually  depict  the 
extent  to  which  the  technological  developments  contribute  to  the 
achievement  of  the  primary  objective  of  the  program. 

The  factors  included  within  this  class  are  those  most  directly 
related  to  the  definition  and  determination  of  the  value  of  the  devel¬ 
opment  program.  For  commercial  enterprises,  these  factors  would  in¬ 
clude  features  of  the  company’s  sales  programs  and  markets  that  would 
determine  the  company’s  revenues  or  profits.  Various  value  measures 
are  frequently  devised  for  Defense  and  other  programs  for  which  no 
market  transactions  occur. 

b.  Operational  Requirements.  The  factors  that  are  classified 

as  operational  requirements  are  those  that  specify  the  various  missions 
that  the  Defense  Department  is  charged  with  fulfilling  and  the  general 
human  and  equipment  capabilities  that  would  contribute  to  those 
missions. 

c.  Weapon  Systems.  Weapon  system  factors  are  those  that  are 
related  to  the  description  of  the  operating  and  performance  character¬ 
istics  contained  in  the  design  of  specific  combinations  of  military 
equipment  and  personnel. 

d.  Technologies.  For  the  purpose  of  the  general  model,  tech¬ 
nologies  are  the  pools  of  skills  and  techniques  that  combine  theoretical 
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and  practical  information  to  devise  processes  and  equipment  for  carry¬ 
ing  out  some  operation.  The  related  factors  are  those  that  set  out 
the  skills,  techniques,  and  information  that  would  be  improved  in  the 
execution  of  the  development  program. 

e.  Development  Tasks.  The  factors  classified  as  part  of  the 
development  tasks  are  those  that  are  concerned  with  specifying  the 
alternative  combinations  of  manpower  and  equipment  and  the  alterna¬ 
tive  approach  strategies  that  might  be  employed  to  improve  the  skills, 
techniques,  and  information  composing  a  technology. 

f.  Costs .  Cost  factors  are  those  that  specify  and  accumulate 
the  real  or  implied  payments  that  the  development  agency  must  make 
for  the  resources  it  employs  in  the  implementation  of  its  program. 

g.  Timing.  Timing  factors  include  those  that  determine  the 
calendar  time  that  will  elapse  with  the  progress  of  the  development 
program.  These  factors  are  generally  related  to  many  of  the  factors 
already  listed.  However,  to  prevent  much  apparent  repetition  in  the 
descriptions  and  analyses  of  the  various  quantitative  methods,  the 
methods  of  including  timing  considerations  are  reviewed  together. 

h.  Risk.  Risk  factors  are  those  that  describe  and  take  into 
account  how  the  actual  outcome  of  part  of  a  development  program  may 
diverge  in  some  aspect  from  what  is  expected  at  the  outset  of  the 
program.  As  is  the  case  with  timing  factors,  risk  factors  are  related 
to  a  number  of  the  factors  already  listed  above  ,  but  they  may  be  treated 
as  a  group  for  expository  purposes. 

4.  Constraints 

In  the  organization  of  a  quantitative  method,  constraints  are 
the  considerations,  described  literally  or  mathematically,  that  are 
basically  external  to  the  immediate  development  process  of  concern 
but  must  be  taken  into  account  in  the  formulation  of  the  development 
program.  Some  considerations  can  be  beyond  the  authority  of  the 
management  of  the  development  program  but  nonetheless  crucial  to  its 
allocation  of  development  resources.  For  example,  the  level  of 
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management  at  which  the  final  allocation  of  resources  is  made  among 
tasks  may  have  some  influence  on  the  total  budget  it  will  receive ,  but 
by  and  large  the  budget  is  decided  at  a  higher  executive  or  legislative 
level.  Consequently,  that  management  must  accept  the  funding  avail¬ 
able  as  a  constraint  on  its  program  formulation. 

5.  Decision  Algorithm 

Within  a  quantitative  method,  the  decision  algorithm  is  the 
formal  procedure  that  is  applied  to  the  allocation  problem,  as  it  is 
formulated  in  the  components  of  the  General  Model  described  above, 
to  choose  the  specific  allocation  that  should  be  implemented. 
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III.  DESCRIPTIONS  OF  QUANTITATIVE  METHODS 

In  this  chapter,  each  of  the  quantitative  methods  finally  chosen 
for  review  in  this  study  is  described  within  the  framework  of  the 
General  Model  developed  in  the  previous  chapter. 

A.  INDUSTRIAL  ANALOG 

Under  contract  to  ARPA,  Research  Analysis  Corporation  performed 
work  during  1969  to  devise  a  quantitative  method  for  planning  resource 
allocation  in  the  Defense  Research  and  Exploratory  Development  Program 
categories  (Refs.  5,  6).  The  proposed  method,  titled  "The  Industrial 
Analog,"  was  not  directed  at  the  valuation  or  consideration  of  indi¬ 
vidual  projects  or  specific  tasks.  Instead,  it  was  aimed  at  aiding 
decision  making  at  a  more  aggregative  level,  proposing  to  determine 
total  expenditures  on  research  and  exploratory  development  by  first 
determining  the  allocation  of  funds  to  a  set  of  component  scientific 
and  engineering  "technical  categories."  Statistical  correlations  were 
to  be  used  in  Industrial  Analog  to  establish  the  funding  of  each 
"technical  category." 

1.  Primary  Objective  of  the  Method 

The  principal  objective  of  the  Industrial  Analog  method  is  to 
determine  the  level  of  funding  for  the  major  technology  fields  within 
the  DoD  6.1  and  6.2  program  categories  (Research  and  Exploratory 
Development)  in  such  a  way  that  the  allocation  of  those  development 
resources  is  similar  to  the  pattern  of  expenditures  made  on  basic  and 
applied  research  by  American  industry. 

The  general  guideline  followed  in  the  formulation  of  the  method 
is  that  the  Defense  Department  might  well  try  in  some  way  to  imitate 
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American  industry  in  determining  the  effort  it  should  expend  upon 
developing  various  technologies. 

The  rationale  for  DoD’s  imitating  American  industry  rests  mainly 
on  the  proposition  that  the  commercial  success  generated  by  industry’s 
R&D  funding  behavior  might  be  translatable  into  national  security 
success  if  DoD  were  to  behave  similarly. 

American  industry  as  a  whole  has  gained  promi¬ 
nence  in  international  markets  for  many  of  its 
products,  and  selected  companies  have  gained  and 
maintained  competitive  advantages  over  other  members 
within  their  industrial  group....  A  selected 
industry- by- industry  analysis  of  the  proportion  of 
company  sales  reinvested  in  basic  and  applied 
research  was  performed  for  the  purpose  of  deter¬ 
mining  the  existence  of  desirable  analogs  or  guide¬ 
lines  for  allocating  the  DoD  resources  (Ref.  6, 
p.  4). 

2.  Control  Variables 

In  the  Industrial  Analog  method,  the  principal  variable  controlled 
by  the  decision  maker  is  the  amount  of  funds  to  be  allocated  to  a  par¬ 
ticular  major  scientific  or  engineering  technical  category.  For  these 
purposes,  the  6.1  and  6.2  program  work  is  grouped  into  nine  such 
technical  fields:  (1)  aircraft,  (2)  missiles,  (3)  combat  vehicles, 

(4)  combat  vessels,  (5)  surveillance,  intelligence  and  target  acquisi¬ 
tion,  (6)  data  processing,  (7)  communications,  (8)  ordnance — nuclear 
and  conventional,  and  (9)  other. 

3.  Factors  and  Their  Relationships 

Because  the  Industrial  Analog  method  is  also  concerned  with  the 
behavior  of  American  industry  in  allocating  development  resources, 
all  the  factors  it  takes  into  account  do  not  fit  directly  into  the 
Defense  terminology  used  in  the  general  model.  However,  Defense 
counterparts  exist  for  many  of  the  concepts  and  measures  that  charac¬ 
terize  American  industry  and  these  parallels  are  utilized  for  organiz¬ 
ing  the  features  of  the  method. 

a.*  Value  Measures.  In  the  Industrial  Analog  method,  no  attempt 

" 

was  made  to  try  to  calculate  the  value  that  the  respective  Research 


18 


and  Exploratory  Development  efforts  might  have  for  the  overall  well¬ 
being  of  either  industry  or  Defense. 

b.  Operational  Requirements.  As  traditionally  conceived,  broad 
mission  statements  are  not  used  for  either  industry  or  Defense  in  the 
Industrial  Analog  method  but  parallel  concepts  are  used  for  classifi¬ 
cation  purposes.  For  the  Defense  Department,  weapon  system  or  sub¬ 
system  categories  serve  this  function.  These  categories  have  already 
been  listed  under  Section  II-B-2  as  the  technical  fields  to  which 
development  funds  are  allocated. 

Industry  groupings  used  in  the  method  can  be  considered  as 
the  commercial  counterparts  to  Defense  missions.  The  groupings 
selected  for  study  were  (Ref.  5,  p.  5): 

Chemical  and  allied  products 
Petroleum  refining  and  extraction 
Aircraft  and  missiles 

Electrical  equipment  and  communications 
Motor  vehicles  and  other  transportation 
Machinery 

Professional  and  scientific  instruments 
Rubber  products 
Primary  metals 
Fabricated  metal  products 

c.  Weapon  Systems.  Specific  weapon  systems  that  had  actually 
been  deployed  by  the  various  Services  and  specific  products  that  had 
been  marketed  by  the  companies  in  the  industry  groupings  were  listed 
and  were  to  be  taken  into  account  in  the  analysis. 

At  the  same  time,  the  analysis  was  to  incorporate  considera¬ 
tion  of  total  expenditures  of  DoD  and  the  net  sales  of  each  industry 
grouping.  These  might  be  taken  to  be  some  measure  of  the  quantity 
of  the  systems  or  products.  However,  it  should  be  emphasized  that 
total  DoD  expenditures  and  net  industry  sales  have  a  broader  scope 
and  a  different  timeframe  than  the  weapons  and  the  commercial  products 
of  interest  in  the  method. 
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d.  Technologies.  As  defined  in  the  general  model,  the  specific 
technology  content  of  the  Defense  weapon  systems,  commercial  products, 
or  the  development  programs  of  either  Defense  or  industry  is  not 
described  in  the  Industrial  Analog  method.  However,  somewhat  relatedly, 
the  method  is  concerned  with  the  lifetimes  of  both  military  weapon 
systems  and  commercial  products.  In  addition,  the  method  proposes 

that  the  lifetimes  of  the  systems  and  products  should  be  related  to 
their  technical  complexities  through  some  functional  representation. 

At  the  stage  to  which  the  study  was  carried,  technical  complexity  was 
not  defined. 

e.  Development  Tasks.  Being  focused  at  a  level  somewhat  above 
the  planning  of  an  individual  project,  the  Industrial  Analog  method 
does  not  trace  or  specify  the  tasks  that  could  be  taken  in  the  develop¬ 
ment  programs  of  the  various  technical  fields. 

f .  Costs.  The  costs  taken  into  account  in  the  Industrial  Analog 
method  are  similarly  aggregative. 

For  industry,  the  annual  company- funded  expenditures  on  R&D 
by  each  industry  group  are  the  principal  measures  of  the  R&D  costs 
incurred.  Using  historical,  time  series  data,  the  method  proposed  to 
relate  the  expenditures  to  annual  net  industry  sales  and  these,  in 
turn,  to  the  corresponding  commercial  product  complexities  and  lifetimes. 

For  DoD,  expenditures  on  R&D  at  three  levels  are  taken  into 
account:  first,  DoD  expenditures  on  each  technical  field  in  the  6.1 
and  6.2  program  categories;  second,  total  DoD  expenditures  on  the  6.1 
and  6.2  program  categories;  and  third,  total  DoD  expenditures  on  RDT&E. 

A  set  of  relationships  of  these  measures  was  also  proposed  for  analysis. 
Time  series  data  covering  annual  DoD  expenditures  on  each  technical 
field  in  the  6.1  and  6.2  program  categories  were  to  be  related, 
respectively,  to  corresponding  total  DoD  expenditures,  total  DoD 
expenditures  on  the  6.1  and  6.2  program  categories,  and  total  DoD 
expenditures  on  RDT&E. 
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g.  Timing .  The  Industrial  Analog  method  does  not  attempt  to 
explain,  as  such,  the  calendar  time  that  has  or  must  elapse  in  the 
pursuit  of  industrial  and  Defense  development  programs. 

h.  Risk.  The  Industrial  Analog  method  does  not  treat  explicitly 
for  either  industry  or  the  DoD  the  possible  divergence  of  the  actual 
results  of  a  development  program  from  the  results  that  were  expected 
at  the  outset  of  the  program. 

4.  Constraints 


No  limitations  on  the  allocations  to  the  different  technical 
fields  arising  from  considerations  external  to  the  development  process 
have  been  included  in  the  formulation  of  the  Industrial  Analog  method. 

5 .  Decision  Algorithm 

No  formal  decision  algorithm  has  been  devised  for  calculating  the 
allocation  of  6.1  and  6.2  funds  to  the  different  technical  fields. 
Derivation  of  the  procedure  for  determining  the  distribution  of  funds, 
using  the  statistical  relationships  described  above  and  additional 
information  such  as  DoD  weapon  system  lifetimes  and  complexities,  is 
listed  as  one  part  of  the  research  that  is  still  required  to  complete 
the  project. 

In  the  work  reported,  some  evidence  is  given  of  the  general 
criteria  that  are  intended  to  be  incorporated  into  the  procedure. 

The  objectives  of  the  study  are  to  (1)  determine 
the  proportion  of  sales  invested  by  selected  segments 
of  U.S.  industry  in  basic  and  applied  research  to 
maintain  competitive  products,  and  (2)  derive  criteria 
and  a  planning  analog  for  making  analytical  compari¬ 
sons  between  6.1  (research)  and  6.2  (exploratory 
development)  defense  expenditures  and  comparable 
U.S.  industry  expenditures.  (Ref.  6,  p.  5) 

Among  the  remaining  research  required  to  develop  the  desired  analog: 

Develop  an  industrial  analog  to  defense  research 
expenditures  based  on  product  competitive  lifetime 
and  complexity.  (Ref.  6,  p.  22) 
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B.  TORQUE 


In  1967  (Ref.  7),  the  Director  of  Defense  Research  and  Engineering 
established  an  inter-Service  ad  hoc  committee  to  investigate  (1) 
methods  for  demonstrating  the  connections  of  current  work  in  the 
Exploratory  Development  program  category  (6.2)  to  future  Defense 
objectives  and  (2)  approaches  to  attaining  a  balanced  allocation  of 
funds  within  the  Exploratory  Development  program.  In  response  to 
its  charter,  the  committee  formulated  a  quantitative  technique  to 
determine  the  distribution  of  the  Exploratory  Development  budget. 

This  technique  is  titled  TORQUE,  for  Technology  or  Research  Quanti¬ 
tative  Utility  Evaluation  (Ref.  8). 

In  part  because  of  common  authors,  TORQUE  closely  resembles 
a  set  of  other  quantitative  analytical  or  allocation  techniques  includ¬ 
ing  BRAILLE  (Ref.  9),  QUEST  (Ref.  10),  and  MACRO  R&D  (Ref.  11). 

In  contrast  to  the  aggregative  allocations  determined  by  a  method 
such  as  the  Industrial  Analog,  TORQUE  focuses  on  determining  the 
allocation  of  funds  to  individual  project  tasks.  To  do  this,  the 
formulation  of  the  method  includes  several  intricate  relationships 
among  a  large  number  of  factors  that  are  used  to  characterize  the 
weapon  system  development  and  utilization  process.  These  relation¬ 
ships,  in  turn,  are  transformed  into  a  set  of  mathematical  expressions 
that  are  used  in  a  formal  computer  program  designed  to  optimize  the 
allocation  of  funds. 

TORQUE  has  been  given  quite  serious  consideration.  Using  the 
Air  Force  Flight  Dynamics  Laboratory  as  a  test  case  in  a  simulated 
annual  program  development  exercise,  an  attempt  was  made  to  assess 
the  problems  that  might  arise  with  its  application  to  actual  budget 
allocation  questions. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  TORQUE  method  is  to  choose  the 
combination  of  development  tasks  and  work  units  that  maximizes  the 
"military  utility”  derived  from  the  weapon  systems  that  result  and 
the  future  military  missions  that  will  be  served  by  these  systems. 
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2.  Control  Variables 


The  principal  control  variables  in  TORQUE  are  the  dollar  expendi¬ 
tures  that  are  to  be  made  in  the  next  year  on  project  tasks  or  work 
units  directed  at  a  particular  technology  objective.  In  determining 
that  allocation,  a  second-order  control  is  established  over  the  weapon 
systems  that  will  be  developed  ultimately  and  the  particular  technical 
route  of  development  that  will  be  followed  for  each. 

3.  Factors  and  Their  Relationships 

The  formulators  of  TORQUE  have  taken  into  account  a  large  number 
of  factors  and  introduced  them  into  a  rather  extensive  structure  of 
relationships.  These  can  be  represented  in  a  tabular  form  such  as 
that  shown  in  Table  2. 

TABLE  2.  TORQUE 
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a.  Value  Measure.  A  special  concept  of  "military  utility" 
has  been  devised  in  the  formulation  of  TORQUE  to  measure  the  value 
of  different  allocations  of  development  resources.  This  concept  is 
applied,  with  some  variations,  to  the  technological,  financial,  and 
time  factors  characterizing  the  different  development  projects  and 
the  missions  and  weapons  to  which  the  projects  contribute.  The 
component  values  are,  in  turn,  transformed  into  a  single  measure  to 
depict  overall  military  usefulness. 

In  the  derivation  of  the  measure,  each  factor  is  evaluated 
in  its  special  context  by  a  group  of  experts  that  applies  its  collec¬ 
tive  judgment  to  assigning  the  factor  a  number  that  reflects  quanti¬ 
tatively  the  criteria  of  the  evaluation.  The  factors,  the  criteria 
of  evaluation,  the  techniques  of  number  assignment,  and  the  integration 
of  all  these  elements  into  the  single  measure  of  value  are  discussed 
in  detail  in  the  following  sections. 

b.  Operational  Requirements.  One  of  the  first  inputs  needed 
for  the  implementation  of  TORQUE  is  the  definition  of  a  set  of 
Operational  Capability  Objectives  (OCOs)  by  the  top  levels  of  manage¬ 
ment  within  each  military  Service,  by  the  Joint  Chiefs  of  Staff,  or 
by  the  Office  of  the  Secretary  of  Defense.  Each  0C0  is  a  major 
mission  that  the  Defense  Department  may  be  expected  to  carry  out. 

It  should  be  a  rather  broad  or  general  operational  function  that 
the  Defense  Department  would  perform  in  support  of  U.S.  national 
security  goals  over  a  long  period  of  time,  independent  of  any  specific 
equipment  that  might  be  used  to  fulfill  it.  These  missions  are 
represented  by  the  x,  y  titles  in  the  column  of  Table  2  labeled 
"0C0." 

Once  the  OCOs  are  defined,  the  same  or  similarly  high- 
ranking  officials  must  assign  to  each  a  numerical  weight  that  corre¬ 
sponds  to  its  relative  importance  among  the  Defense  missions.  These 
weights  are  the  W^s  of  the  "utility  function"  illustrated  in  the 
column  of  Table  2  labeled  "0C0  Weight."  They  reflect  the  value 
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schemes  of  the  officials  who  must  determine  them  in  the  light  of 
their  conceptions  of  the  national  security  goals. 

To  ensure  the  internal  consistency  of  the  0C0  weights, 
the  TORQUE  manuals  describe  and  recommend  application  of  the  Churchman 
Ackoff  value  measurement  technique  (Ref.  12,  pp.  87-91).  This  tech¬ 
nique  was  devised  as  a  means  of  assigning  approximate  measures  of 
value  to  various  events.  In  order  that  the  value  of  each  event  can 
be  assigned  by  this  technique  the  events  must  be  independent.  That 
is,  the  events  can  occur  separately  or  in  any  combination,  and  if 
they  do  occur  in  a  combination  the  value  that  would  be  attributed 
to  the  combination  is  the  sum  of  the  values  assigned  to  each  event 
as  though  it  were  to  occur  by  itself. 

c.  Weapon  Systems.  In  the  TORQUE  procedure,  once  the  OCOs  are 
defined,  an  inter-disciplinary  team  (I.D.  team),  composed  of  users, 
technologists,  and  systems  analysts,  devises  alternative  weapon 
systems  or  equipment- tactic  concepts  for  carrying  out  the  missions. 

As  illustrated  in  Table  2,  by  System  B,  some  of  the  systems  may  well 
be  used  in  more  than  one  mission.  An  example  would  be  an  aircraft 
that  could  be  used  in  both  strategic  bombing  and  tactical  interdiction 

The  precise  procedures  that  the  I.D.  team  should  follow 
are  not  spelled  out  in  the  manuals.  The  interaction  of  the  personnel 
on  the  team  determines  the  nature  of  its  deliberations,  the  types  of 
systems  it  will  propose,  the  number  of  systems  it  will  eventually 
incorporate  into  the  TORQUE  calculations  for  each  0C0,  and  the 
criteria  it  will  use  to  set  the  types  and  number. 

d.  Technologies.  Using  intelligence  estimates  and  other 
information  from  long-range  plans,  the  I.D.  team  makes  a  very  rough 
design  of  each  candidate  weapon  system  serving  the  various  OCOs. 

From  these  designs,  the  team  further  establishes  the  technology 
requirements  of  the  systems.  For  example,  the  team  may  make  a  rough 
design  of  a  long-haul  airlift  aircraft  and  then  derive  specific 
technology  objectives  for  the  propulsion  system,  the  navigation 
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system,  and  the  landing  gear,  as  well  as  for  several  other  technol¬ 
ogies,  in  fairly  specific  quantitative  terms. 

The  team  also  must  evaluate  the  ,f criticality"  of  each 
technology  objective  to  the  weapon  systems.  Definitions  of  the  levels 
of  "criticality"  and  the  numerical  values  assigned  to  the  factors 
representing  these  levels  are  given  in  Table  3.  In  Table  2,  the 
"criticality"  values  are  entered  in  the  cell  at  the  intersection  of 
the  system  row  in  each  0C0  block  with  the  column  for  the  same  system 
under  the  technology  objective  heading.  For  example.  Technology 
Objective  1  has  a.  "criticality"  of  0.3  in  Weapon  System  B  when  it  is 
employed  in  0C0  x  but  0.7  when  it  is  employed  in  0C0  y.  For  a  type 
of  aircraft  that  might  be  used  in  strategic  and  tactical  missions, 
Technology  Objective  1  might  be  thought  to  apply  to  a  component  that 
affects  low-altitude  maneuverability.  On  the  other  hand,  Technology 
Objective  2  might  be  related  to  the  operation  of  an  extremely  accurate 
navigational  system. 

e.  Development  Tasks.  A  technology  team  composed  of  scientific 
and  engineering  personnel  determines  whether  some  of  the  technology 
objectives  supporting  the  various  weapon  systems  are  sufficiently 
alike  to  be  treated  as  a  single  technology  objective  (such  as  Tech¬ 
nology  Objective  1  in  Table  2  being  common  to  systems  A,  B,  and  D) . 

This  team  also  determines  whether  a  group  of  technology  objectives 
may  fit  into  a  single  development  sequence  within  which  the  accom¬ 
plishment  of  some  may  be  necessary  for  developing  others. 

f.  Costs.  Once  the  technology  team  has  sorted  out  the  technology 
objectives  from  the  various  weapon  systems,  it  proceeds  to  estimate 
the  time- phased  funding  that  will  be  required  for  the  development  of 
each  objective.  The  timing  features  that  are  considered  in  these 
estimates  are  described  more  fully  in  the  following  paragraphs. 

g.  Timing .  Given  the  development  of  the  threat,  current  system 
capabilities  and  current  procurement  and  phase-out  plans,  the  I.D. 
team  must  estimate  the  earliest  date  by  which  a  new  system  might  be 
introduced  into  the  force  and  the  latest  date  by  which  it  could  be 
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TABLE  3.  CRITICALITY  OF  A  TECHNOLOGY  OBJECTIVE 
TO  A  WEAPON-SYSTEM/OCO  COMBINATION 


(The ' assumption  is  that  the  objective  of  the  technical  effort  will  be 
accomplished. ) 

Absolutely  Essential 

Failure  to  have  this  technology  will  absolutely  prevent 

the  attainment  of  the  capability  desired . 1.0 

Major  Contribution 

Failure  to  acquire  this  technology  will  result  in  a 

significant  decrease  in  one  or  more  of  the  major 

performance  parameters  needed  to  attain  the  capability 

desired.  Such  degradation  probably  would  not  prevent 

a  favorable  decision  for  development  of  equipment  for 

the  invent ory . 0.7 

Cost  Reduction 

Success  in  achieving  this  technology  will  provide  a 

major  reduction  in  the  cost  of  achieving  the  capability 

desired. . . 0.5 

Substantial  Contribution 

Failure  to  achieve  this  technology  will  result  in  the 
loss  of  a  highly  desirable  but  not  essential  capability. 

Such  degradation,  while  important,  probably  would  not 
prevent  a  favorable  decision  on  the  development  of  equip¬ 


ment  for  the  inventory  to  attain  the  capability  desired . 0.4 

Refinement  of  Capability 

Achievement  of  this  technology  will  result  in  some 

refinement  of  the  present  capability.  The  desired 

capability,  however,  could  be  achieved  without  this 

effort . 0.3 

Indirect  Contributions 

Achievement  of  this  technology  will  only  be  an  indirect 
contribution  to  the  capability  desired . 0.2 

Remote  Association 

This  effort  has  only  a  remote  association  with  the 

capability  desired . . . 0.1 

No  Contribution . 0.0 


Source:  Ref.  8. 


A 
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introduced  without  loss  of  effectiveness.  For  the  illustrations  in 
Table  2,  the  initial  operating  capability  dates,  in  calendar  years, 
are  listed  in  the  column  titled  "Systems  IOC  CY." 

Consideration  of  the  system  IOC  dates  must  feed  back  onto  the 
I.D.  team  choice  of  systems  to  be  proposed.  There  would  be  little 
point  in  proposing  an  "obsolete"  system  or  a  system  that  could  not  be 
introduced  before  the  effectiveness  of  the  preceding  system  in  the 
force  deteriorated  significantly. 

Subsequently,  the  I.D.  team  must  estimate  the  dates  by  which 
attainment  of  the  technology  objective  is  needed  to  support  the  system 
IOC  dates.  The  need  dates  depend  upon  the  lag  between  the  date  the 
technology  is  demonstrated  in  Exploratory  Development  and  the  date 
it  can  finally  be  incorporated  into  actual  production  for  procurement 
in  a  new  weapon  system.  In  Table  2,  Technology  Objective  1  must  be 
demonstrated  by  1976  at  the  earliest  and  1977  at  the  latest  to  permit 
Weapon  System  A  IOC  dates  of  1981  or  1982. 


In  line  with  the  I.D.  team  determinations,  the  technology  team 
estimates  a  time-phased  budget  for  each  weapon  system  application  of 
a  technology  objective,  the  pattern  of  annual  funding  set  so  that  the 
completion  of  the  technology  objective  will  match  the  latest  need  date 
for  that  application.  These  are  illustrated  in  Table  2  by  the  entries 
F  ^  in  the  last  rows  of  the  table.  For  example,  ^Bl  t*"ie  fading 
of  Technology  Objective  1  that  will  be  required  in  the  second  year  to 
develop  the  technology  objective  in  time  to  meet  the  1978  need  date 
for  Weapon  System  B. 


In  TORQUE  the  assumption  is  made  that  a  technology  objective  is 
not  worth  its  full  "utility"  in  a  particular  weapon  system  application 
if  it  is  completed  outside  the  interval  of  its  earliest  and  latest 
need  dates  for  that  application.  This  assumption  reflects  two  con¬ 
siderations.  If  the  technology  objective  is  developed  before  the 
early  need  date,  the  resources  used  in  its  development  might  have 
been  better  used  in  some  other  effort.  If  the  technology  objective 
is  developed  after  the  late  need  date,  the  effectiveness  of  the  system 
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it  supports  will  have  deteriorated.  The  analytical  form  of  the 

assumption  used  in  TORQUE  is  the  Timeliness  Function,  t?.,  ,  illus- 

1 3  k 

trated  in  Fig.  2. 


E  =  EARLIEST  NEED  DATE 
L  =  LATEST  NEED  DATE 


FIGURE  2.  TORQUE  Timeliness  Function 


The  value  of  t5  depends  basically  upon  the  funding  devoted  to 
the  technology  objective  in  the  next  year,  F-^.  For  any  F-^,  an 
estimate  is  made  of  the  completion  date  of  the  resulting  development 
program  for  the  technology  objective.  In  general,  if  F^  is  less 
than  the  first  year  funding  in  the  time-phased  budget  originally 
estimated  by  the  technology  team  for  the  particular  application, 

F^.^,  the  completion  date  will  be  set  back  from  target  date  used  by 
the  team.  If  F^  is  greater  than  the  first  year  funding  in  the  team’s 
original  budget,  F.  ,  the  completion  date  will  be  moved  forward  from 

l  j  k. 

the  target.  The  completion  date,  in  turn,  is  compared  with  the  tech¬ 
nology  objective  need  dates  specified  for  the  particular  weapon  system 

F 

and  the  value  of  t^ is  then  set  according  to  the  trapezoid-shaped 
relationship  shown  in  Fig.  2. 
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h.  Risk.  The  formulation  of  TORQUE  does  not  take  into  account 
that  the  actual  progress  in  technical  performance,  timing,  or  cost  of 
a  development  effort  contributing  to  a  technology  objective  may  deviate 
from  the  projections  made  of  these  variables  at  the  time  the  allocation 
of  resources  is  made.  The  development  process  is  treated,  more  or 
less,  as  precisely  predictable. 

4.  Constraints 

In  TORQUE  the  total  budget  made  available  to  the  program  is  the 
principal  consideration  external  to  technology  development  that  is 
imposed  upon  the  choice  procedure.  However,  as  explained  in  the 
experimental  test  manuals  (Ref.  8),  the  set  of  candidate  weapon  sys¬ 
tems  incorporated  into  the  TORQUE  choice  framework  is  specified  before¬ 
hand,  apparently  on  the  basis  of  considerations  outside  those  taken 
directly  into  account  in  TORQUE.  Consequently,  the  choice  of  systems 
made  by  TORQUE  in  allocating  resources  to  the  Exploratory  Development 
program  is  also  constrained. 

5.  Decision  Algorithm 

All  of  the  factors  and  relationships  that  have  been  described  are 
components  of  the  terms  in  the  ,rutility  function.” 


2  W.C. . 

i  1  XD 


where  U 

F 

u. 


W. 

i 


=  total  "military  utility,"  or  overall 
"utility  function" 

=  "utility"  of  work  on  technology  objective  k  when 

funded  at  level  next  year,  the  technology  objective 
"utility  function" 

=  relative  weight  of  operational  capability  objective  i, 
the  basic  measure  of  value 
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and 


=  the  "criticality"  of  technology  objective  k  for  weapon 
system  j  when  employed  in  operational  capability 
objective  i 


*Fk 


lk 


£  F, 


yk 


?lk  =  ^°-^ar  funding  of  work  in  technology  objective  k 
next  year,  the  control  variable 

F  ^  =  dollar  funding  of  work  in  technology  objective  k 

in  year  y 
F 

t^k  =  the  timeliness  function  value  for  technology  objective 
k  in  system  j  applied  to  operational  capability  objec¬ 
tive  i  when  funded  at  F^  next  year. 


As  defined  above,  the  utility  function  is  composed  of  the  sepa¬ 
rate  utility  functions  of  the  individual  technology  objectives.  How¬ 
ever,  each  of  these  technology  objective  utility  functions  is  further 
decomposable  into  more  basic  utility  functions  defined  by  the  relation¬ 
ship  of  the  technology  objective  to  a  specific  weapon  system  with  the 
funding,  Fn,  ,  variable.  In  each  of  these  more  basic  utility  functions, 
and  are  constants;  and  t^.^  vary  with  F^.  varies 

from  0  to  1  as  F,,  varies  from  0  to  S  F  .  .  However,  t?..  varies  from 
lk  y  yk  ’  ljk 

0  to  1  and  back  to  0  as  Fnl  increases. 


Consequently,  the  basic  utility  functions,  technology  objective 
utility  functions,  and  the  general  "utility  function"  can  be  made 
dependent,  in  the  first  instance,  upon  the  quantity  of  funds  spent 
in  the  relevant  part  of  the  development  program.  A  basic  utility 
function  will  generally  have  a  single  peak  or  maximum  but  the  tech¬ 
nology  objective  utility  function,  formed  by  aggregating  basic 
utility  functions,  may  assume  quite  irregular  shapes. 


Besides  being  a  component  of  the  utility  function  the  cost  re¬ 
lationship,  which  depicts  the  funding  requirements  of  the  progress  on 
a  technology  objective,  is  used  separately  in  the  choice  procedure  and 
the  budget  tabulations  of  the  algorithm. 
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A  very  specific  algorithm  has  been  devised  in  TORQUE  to  determine 
the  distribution  of  the  program  budget  among  the  candidate  technology 
objective  tasks  and  work  units.  Basically,  the  algorithm  tries  to 
take  into  account  simultaneously  the  contributions  of  the  technology 
objectives  to  the  utility  function,  the  costs  of  the  technology  objec¬ 
tive  contribution  to  the  utility  function,  and  the  overall  program 
budget  constraint. 

To  some  extent,  the  algorithm  resembles  a  Lagrangian  multiplier 
maximization  technique  in  the  solution  to  which  the  incremental  con¬ 
tribution  of  utility  would  be  approximately  the  same  from  the  expendi¬ 
ture  of  the  last  dollar  in  each  technology  objective.  In  other  words, 
it  attempts  to  allocate  total  available  funds  among  the  technology 
objectives  in  such  a  way  that  any  reallocation  of  funds  among  the 
technology  objectives  would  not  increase  the  value  of  the  utility 
function.  The  algorithm  performs  the  allocation  in  a  sequential 
fashion  (described  in  the  Appendix)  assigning  additional  units  of 
funds  to  the  various  technology  objectives  in  the  order  of  descending 
increments  of  utility  per  additional  dollar  expenditures  within  and 
across  all  technology  objectives. 

6.  Special  Aspects 

Following  from  the  nature  of  the  control  variables  and  the  cri¬ 
teria  considered  in  TORQUE,  the  resulting  solution  displays  the  total 
funds  that  should  be  spent  on  each  Exploratory  Development  project 
task  or  work  unit  during  the  next  program  year.  According  to  the 
TORQUE  manual,  that  solution  could  then  be  used  in  the  followup  exercise, 
stepping  up  by  one  year  the  budget  proposals  of  the  technology  team, 
to  determine  similarly  the  ’’optimum"  allocation  of  funds  in  the  subse¬ 
quent  year.  This  procedure  can  be  extended  farther  into  the  future 
to  generate  a  time-phased  distribution  of  funds  for  Exploratory  Devel¬ 
opment,  tracing  out  the  later  program  impacts  of  the  current  and  near- 
term  funding  decisions. 
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C.  NAVAL  ORDNANCE  LABORATORY  METHOD 


In  1968,  under  a  task  from  the  Exploratory  Development  Division 
of  the  Naval  Material  Command,  the  Naval  Ordnance  Laboratory  under¬ 
took  to  devise  a  method  for  planning  the  distribution  of  Exploratory 
Development  resources  among  the  various  technologies.  The  NOL  study 
built  upon  work  already  performed  on  the  Navy  Exploratory  Development 
Goals  (EDGs)  and  the  Navy  Technological  Forecast  (NTF)  to  formulate 
a  quantitative  system  for  planning  the  Exploratory  Development  pro¬ 
gram  (Ref.  13). 

Like  TORQUE,  the  NOL  method  is  made  up  of  relationships  among  a 
number  of  factors,  including  military  missions,  weapon  system  con¬ 
figurations,  technology  efforts,  and  costs.  These  relationships  are 
formulated  into  a  mathematical  model  to  which  a  formal  decision 
algorithm  is  applied  to  determine  the  desired  allocation  of  develop¬ 
ment  resources  among  the  various  technologies. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  NOL  method  is  to  allocate  the  devel¬ 
opment  budget  among  the  various  technologies  to  produce  the  Explora¬ 
tory  Development  program  having  the  maximum  value  for  future  needs 
of  the  Navy.  Value  and  the  future  needs  of  the  Navy  have  specific 
meanings  for  the  purposes  of  the  method.  These  meanings  are  described 
in  some  detail  below. 

2.  Control  Variables 

In  the  NOL  method,  the  principal  control  variable  of  the  manager 
is  the  quantity  of  funds  that  will  be  devoted  to  work  in  a  particular 
technology  during  the  next  budget  year.  As  in  TORQUE,  however,  the 
funding  exercises  much  broader  second-order  control  over  the  types  of 
weapon  systems  that  will  be  ultimately  developed  and  the  particular 
missions  that  will  be  emphasized. 
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3.  Factors  and  their  Relationships 


The  NOL  method  considers  a  number  of  factors  in  devising  its 
allocation  of  development  resources  among  the  technologies.  These 
factors  are  incorporated  into  relationships  that  the  formulators 
have  devised  to  describe  the  interactions  of  the  factors  and,  ulti¬ 
mately,  their  effects  in  the  real  development  process. 

a.  Value  Measure.  "Value"  is  attributed  to  each  technology 
competing  for  funds  in  the  method.  However,  the  value  of  any  tech¬ 
nology  is  derived  from  the  values  attached  to  a  set  of  factors  that 
is  treated  as  more  basic  for  development  planning.  For  these  pur¬ 
poses,  value  is  measured  on  an  arbitrary  scale,  using  a  number  system 
to  depict  the  relative  rankings  of  the  technologies  and  more  basic 
factors.  The  derivations  are  described  more  fully  below  under  the 
various  factors  that  are  taken  into  account  and  the  overall  synthesis 
of  the  factors  for  determining  the  budget  allocation. 

b.  Operational  Requirements.  A  special  hierarchy  of  missions 
and  functions  is  used  in  the  NOL  method  to  define  its  counterpart 
concept  to  operational  requirements.  The  three-level  classification 
scheme  has  the  following  structure: 

A.  Warfare  Categories 

1.  Strategic 

2.  Conventional 

3.  Limited  and  Counterinsurgency 

B.  Target /Support  Categories 

1.  Air  Target 

2.  Sea  and  Land  Surface  Target 

3.  Undersea  Target 

4.  Information  Support 

5.  Logistics  and  Ancillary  Support 

C.  Military  Function  Categories 

1.  Command  and  Control 

2.  Target  Data  Collection 

3.  Counteraction 

4.  Mobility 

5.  Support 
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An  operational  requirement  is  described  in  terms  of  a  combination 
of  elements  taken  from  these  categories.  For  example,  one  might  be 
a  close -support  capability  (military  function)  against  sea  and  land 
surface  targets  (target /support)  in  a  conventional  war  (warfare). 

Values  of  these  operational  requirements  are  synthesized  from 
weights  assigned  to  each  of  the  elements  within  the  three  categories. 
These  weights  are  determined  through  a  polling  procedure  much  like 
that  recommended  in  TORQUE,  and  also  employing  the  concept  of  an 
approximate  measure  of  "utility"  proposed  by  Churchman  and  Ackoff 
(Ref.  12). 

The  value  of  an  operational  requirement  can  be  represented  in  the 
following  fashion : 


w  ,  =  w  w,  w  , 

abc  a  bra  c :ab 


where  w^  =  the  value  of  an  operational  requirement  composed 
of  military  function  c,  target /support  b, 
warfare  a 


w  =  the  weight  assigned  to  warfare  category  a,  based 
a  on  its  importance  in  supporting  basic  national 

objectives  in  the  time  period  under  consideration 

w.  =  the  weight  assigned  to  target /support  category  b, 
,a  based  on  its  contribution  to  the  objectives  of 
warfare  type  a 

w  k  =  the  weight  assigned  to  military  function  c,  based 
on  its  importance  to  target /support  category  b 
in  warfare  type  a. 

c.  Weapon  Systems.  Descriptions  of  weapon  system  capabilities 
are  contained  in  the  Exploratory  Development  Goals  (EDGs)  employed  in 
the  NOL  method.  Specifically,  the  EDGs  describe  the  work  that  must 
be  done  at  the  Exploratory  Development  level  to  satisfy  the  operational 
requirements  of  future  weapons  and  support  systems.  They  translate 
the  broad  and  intentionally  general  statements  made  in  long-range 
planning  documents  into  specific,  quantitative  terms  that  will  facili¬ 
tate  planning  of  Exploratory  Development  programs  responsive  to  Navy 
needs.  For  example,  the  EDG  for  a  surface  close-support  capability  to 
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be  employed  against  sea  and  land  surface  targets  will  describe  with 
some  precision  several  quantitative  performance  measures,  including 
the  required  CEP,  the  delivery  range,  and  damage  criteria.  (Ref.  13, 

p.  2-2). 

In  turn,  the  EDG  is  assigned  a  weight  in  a  manner  similar  to  that 
already  described  for  the  operational  requirements  above,  based  upon 
its  "importance"  (relative  to  other  EDGs)  to  the  military  function 
it  serves.  If  this  weight  for  EDG  j  is  designated  w_.  ,  the 

"military  worth"  (value)  of  EDG  j  can  be  represented  in  the  following 
way. 

Wj  =  Wj .abcwabc  =  the  "military  worth"  of  EDG  j 

d.  Technologies.  Basic  technologies  are  spelled  out  in  some 
detail  in  the  NOL  method.  The  areas  of  technology  devised  for  the 
Navy  Technological  Forecast  have  been  adopted  for  these  purposes. 

Each  technology  is  depicted  in  terms  of  a  "pacing  parameter. "  A 
pacing  parameter  is  generally  a  measure  that  has  been  devised  to  com¬ 
municate  readily  an  operational  feature  of  a  technology.  For  example, 
within  rocket  propulsion  technology,  the  specific  impulse  of  a  rocket 
propellant  measures  the  impulse  that  can  be  theoretically  generated 
from  the  reaction  of  that  specific  combination  of  a  fuel  and  oxidizer. 
It  is  stated  in  terms  of  the  number  of  seconds  over  which  one  pound 
of  the  propellant  will  generate  one  pound  of  thrust. 

Technologies  are  linked  to  the  EDGs  in  two  steps.  First,  a 
general  functional  analysis  is  made  of  each  weapon  system  described 
in  the  EDGs.  Second,  expert  judgment  is  applied  to  devising  the  tech¬ 
nological  content  of  each  general  functional  area,  in  terms  of  the 
areas  of  technology  contained  in  the  Navy  Technological  Forecast. 

Once  the  linkages  between  the  technologies  and  EDGs  are  estab¬ 
lished,  the  expert  judgment  is  used  again  to  establish  the  "utility" 
of  each  technology  to  each  EDG.  In  this  case,  utility  is  intended  to 
be  a  measure  of  the  relevance  of  the  effort  required  to  advance  the 
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technology  to  satisfy  the  EDG.  The  expert  (1)  "predicts"  whether 
the  state  of  the  technology  (the  pacing  parameter)  will  be  adequate 
to  meet  the  functional  requirements,  (2)  assigns  numerical  values  to 
describe  the  conditions  of  adequacy,  and  (3)  "normalizes"  these 
values  for  each  EDG  to  show  the  relative  importance  of  work  in  each 
technology  for  the  EDG. 


The  numerical  values  that  describe  the  adequacy  of  a  technology's 
pacing  parameter  to  the  functional  requirement  of  an  EDG  are  called 
"relevance  numbers."  These  are  denoted  by  r^  ,  the  adequacy  of  tech¬ 
nology  i  for  the  functional  requirements  of  EDG  j. 


r^  .  =  4,  if  the  forecast  state  of  technology  i  is  less  than 
required  for  EDG  j, 


=  2,  if  the  forecast  state  of  technology  i  equals  that 
required  for  EDG  j , 

=  1,  if  the  forecast  state  of  technology  i  exceeds  that 
required  for  EDG  j , 

=  0,  if  technology  i  is  not  related  to  EDG  j. 


Finally,  the  "utility"  of  technology  i  to  EDG  j  is  derived  by 
normalizing  the  relevance  numbers  of  the  various  technologies  that 
apply  to  that  EDG. 


u .  . 


=  the  utility  of  technology  i  to  EDG  j . 


e.  Development  Tasks.  The  NOL  method  does  not  explicitly  deal 
with  the  different  combinations  of  manpower  and  equipment  or  the 
different  approaches  that  might  be  used  to  advance  the  pacing  param¬ 
eter  of  a  technology.  However,  it  does  contain  a  rudimentary  concept 
of  the  development  task  insofar  as  it  admits  that  the  pacing  parameter 
can  be  pushed  to  different  levels,  depending  upon  the  effort  expended 
on  the  pacing  parameter. 
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f.  Costs .  Cost  components  and  cost  estimating  techniques  are 
not  treated  in  detail  in  the  formulation  of  the  NOL  method.  The 
general  approach  adopted  relates  the  funding  of  a  technology  to  the 
progress  that  would  be  made  in  the  pacing  parameter  of  that  technology 
over  the  funded  time  period,  without  regard  to  the  actual  resources 
employed. 

To  describe  this  relationship,  an  "Index  of  Advance"  has  been 
devised  for  each  technology.  Basically,  this  index  measures  the 
progress  that  would  be  made  in  the  pacing  parameter  of  the  technology 
under  different  funding  levels  during  the  next  budget  year  relative 
to  the  theoretical  maximum  progress  that  would  be  made  if  unlimited 
funds  were  available. 


(IA).(X.) 


Advance  in  parameter  i  during  plan  time  with  funding  X^ 
Advance  in  parameter  i  during  plan  time  with  no  fund  limit 


An  exponential  approximation  to  this  empirical  relationship  is 
used  in  the  NOL  method. 

(IS)i(X.)  =  1  -  e'Xi/Di 

An  illustration  of  an  index  of  advance  is  shown  in  Fig.  3  along 
with  the  exponential  approximation.  To  follow  the  example  cited 
above,  the  index  might  be  taken  to  show  the  specific  impulses  that 
might  be  developed  in  new  propellants  resulting  from  different  levels 
of  expenditures  on  Exploratory  Development  in  this  technology  rela¬ 
tive  to  the  specific  impulse  that  might  be  attained  with  unlimited 
expenditures  in  the  technology. 

g.  Timing.  The  timing  of  the  EDGs  and  the  development  efforts 
that  must  be  performed  for  their  fulfillment  are  not  treated  explicitly 
in  the  NOL  method.  Consideration  of  calendar  and  elapsed  time  must  be 
assumed  to  be  made  with  the  specification  of  the  EDGs,  their  func¬ 
tional  analyses,  and  technological  links.  The  most  direct  reference 
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to  time  is  contained  in  the  index  of  advance,  which  measures  the 
progress  that  will  be  made  in  a  particular  pacing  parameter  within  the 
following  budget  year  as  a  result  of  the  funding  that  will  be  devoted 
to  that  parameter  during  the  same  time. 


X.  =  Dollar  expenditure  on  technology  i  in  next  year 

D.  =  The  dollar  expenditure  on  technology  i  in  the  next 
year  that  would  result  in  the  pacing  parameter 
progressing  to  a  level  63%  of  the  total  progress 
possible  with  unlimited  funding. 


FIGURE  3.  Simulation  of  a  Typical  Index  of  Advance  Curve  Using  an 
Exponential  Function 

h.  Risk.  No  explicit  consideration  is  given  to  the  possible 
deviations  that  might  occur  between  the  projected  quantities  and  actual 
outcomes  of  the  pacing  parameters,  costs  and  time  requirements  of  the 
development  effort  depicted  in  the  NOL  method. 

4.  Constraints 

The  principal  constraint  considered  in  the  formulation  of  the  NOL 
method  is  the  maximum  total  budget  that  will  be  available  to  be  spent 
on  the  relevant  development  effort  over  the  next  funding  year. 
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5.  Decision  Algorithm 


The  components  of  the  analysis  in  the  NOL  method,  described  above, 
are  integrated  into  a  "Payoff  Function"  that  measures  the  value  of  any 
distribution  of  funds  among  the  technologies,  subject  to  a  budget 
constraint. 


P(X) 


where  P(X) 
(IW). 


=  E  (IW)  (IA)  (X  ) 
i 

<;  C 

=  the  total  Payoff  or  "Value," 

=  the  "Index  of  Worth"  of  technology  i, 


=  the  "Military  Worth"  of  EDG  j , 

=  the  "Utility"  of  technology  i  to  EDG  j . 

(IA)^(X^)  =  the  "Index  of  Advance"  of  technology  i,  a 
function  of  X  , 

X^  •  =  the  amount  of  funds  allocated  to  Exploratory 

Development  in  technology  i,  during  the  next 
planning  year, 

C  =  the  total  budget  available  for  Exploratory 

Development  during  the  next  planning  year. 

To  determine  the  allocation  of  funds  among  the  various  tech¬ 
nologies  that  has  the  maximum  value  for  Navy  needs,  the  NOL  method 
applies  an  algorithm  resembling  a  Lagrangian  multiplier  maximization 
technique  to  the  payoff  function  and  budget  constraint.  Consequently, 
the  distribution  of  funds  is  determined  in  a  way  that  the  last  dollar 
increment  allotted  to  a  technology  in  the  Exploratory  Development  pro¬ 
gram  generates  the  same  increment  in  the  payoff  function  as  the  last 
dollar  increment  alloted  to  any  other  technology. 
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D.  AIR  FORCE  FLIGHT  DYNAMICS  LABORATORY  METHOD 


Beginning  in  1962,  the  planning  staff  at  the  Air  Force  Flight 
Dynamics  Laboratory  (FDL)  developed  a  quantitative  method  to  assist 
in  allocating  the  Lab's  Exploratory  Development  budget  among  the  pro¬ 
posed  tasks.  In  its  formulation,  the  staff  has  incorporated  a  number 
of  factors  and  analytical  features  to  establish  the  value  of  the 
various  tasks.  The  approach  imputes  these  values  both  through  the 
weapons  systems  that  the  tasks  might  eventually  support  and  through 
a  set  of  more  basic  technical  goals  that  are  not  directly  associated 
with  projected  weapons  systems. 

Originally,  the  value  estimates  were  incorporated  into  a  linear 
programming  model  through  which  the  desired  allocation  of  the  budget 
among  the  tasks  was  derived.  According  to  reports  on  the  method, 
experimental  solutions  were  obtained  shortly  after  the  formulation 
work  began  and,  with  additional  development,  solutions  have  been 
derived  for  a  wide  range  of  management  problems.  (Refs.  14,  15) 

Interestingly  enough,  the  Air  Force  Flight  Dynamics  Laboratory 
was  the  site  of  the  principal  trial  that  has  been  run  on  the  TORQUE 
allocation  method. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  FDL  method  is  to  choose  the  levels 
of  effort  that  should  be  expended  upon  the  different  proposed  Explora¬ 
tory  Development  tasks  to  produce  the  maximum  military  value  to  the 
Air  Force.  The  definition  of  military  value  and  its  measurement  for 
each  task  are  treated  in  detail  below. 

2 .  Control  Variables 

The  principal  control  variables  in  the  FDL  method  are  the  various 

l 

amounts  of  resources,  dollars,  and  types  of  engineering  manpower  that 
are  to  be  devoted  to  each  Exploratory  Development  task.  These  re¬ 
sources  are  treated  in  packages  of  discrete  quantities  that  can  be 
applied  to  each  task,  but  linear  combinations  of  these  discrete  quanti¬ 
ties  are  also  admissible. 
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3.  Factors  and  Their  Relationships 


a.  Value  Measure.  Like  TORQUE  and  the  NOL  method,  the  FDL 
method  employs  the  Churchman -Ac koff  approximate  measure  of  value  as 
its  basic  value  concept  (Ref.  12).  In  this  particular  application, 
the  measure  is  used  as  a  quantitative  expression  of  the  "effective¬ 
ness"  of  the  various  development  tasks.  Rankings  of  technological, 
timing,  and  mission  factors,  based  on  various  criteria,  are  trans¬ 
formed  through  a  judgmental  process  into  numbers  that  depict,  on  a 
continuous  scale,  the  value  of  the  tasks  to  the  Air  Force  in  its 
role  supporting  national  objectives. 

b.  Operational  Requirements.  The  formulators  of  the  FDL  method 
used  the  following  scheme  of  wars  and  operations  to  define  the  perti¬ 
nent  operational  requirements  of  the  Air  Force. 

Wars 

General 

Limited 

Cold 

Operations 

Combat 

Reconnaissance 
Logistics 
Show  of  Force 

A  particular  operation  in  the  context  of  a  single  type  of  war  is 
treated  as  a  separate  mission  or  operational  requirement. 

The  Churchman -Ackoff  procedure  is  employed  to  establish  the 
value  of  an  operational  requirement  for  the  purposes  of  Exploratory 
Development.  The  number  associated  with  operation  type  j  in  war  type 
i,  indicates  the  relative  technological  improvement  that  is  needed 

to  support  that  type  of  operation  in  the  specific  war  context. 

c.  Weapon  Systems.  The  range  of  weapon  systems  considered  in 
the  FDL  method  consists  of  the  future  conceptual  Air  Force  flight 
vehicle  systems  proposed  in  the  Air  Force  Systems  Command  Long-Range 
Technological  War  Plan.  A  value  is  assigned  to  each  weapon  system 
corresponding  to  the  support  it  should  provide  each  mission  relative 
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to  the  other  weapon  systems.  These  values,  R™. ,  reflecting  the  rela¬ 
tive  contribution  of  system  m  to  the  ij  mission,  are  derived  by  the 
same  Churchman- Ac kof f  approximate  value  procedure.  In  turn,  the  sys¬ 
tem  mission  values  are  combined  to  form  an  overall  system  value. 

S  =  H  w. .Rm.  =  the  raw  value  of  system  m 
m  ..  i]  i] 

where  w^ .  equals  the  mission  values  described  above. 

Finally,  the  system  values  are  normalized  so  that 


E  S  =  1. 

m 

m 


d.  Technologies.  The  technological  composition  of  each  weapon 
system  is  specified  and  connected  to  the  technological  accomplishments 
that  are  sought  in  the  specific  Exploratory  Development  tasks  proposed 
for  the  FDL  program.  The  connection  is  quantified  in  the  applica¬ 
bility  factor,  Dnm,  which  ranges  from  0.1  to  1.0  depicting  the  appli¬ 
cability  of  the  technology  in  task  n  to  weapon  system  m. 


However,  in  addition  to  the  weapon -systems-oriented  path  to  the 
valuation  of  a  development  task  described  above,  the  FDL  method  in¬ 
cludes  a  technology-oriented  valuation  scheme  as  well.  Technical 
goals,  defined  in  the  Research  and  Technology  Long-Range  Plan,  are 
related  directly  to  the  various  missions  without  reference  to  any 

weapon  systems  in  which  they  eventually  might  be  embodied.  A  tech¬ 
ie 

nical  goal,  k,  is  assigned  a  value  ,  reflecting  its  contribution 
to  fulfillment  of  the  ij  mission  relative  to  the  contributions  of 
other  technical  goals  to  that  mission.  These  values  are  combined  to 
devise  an  overall  value  for  the  technical  goal. 


EE  w. .G.  . 

ij  13  1] 


=  the  overall  value  of  technical  goal  k, 


where  w. .  is  the  mission  values  described  above. 

13 
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e.  Development  Task.  The  development  task  is  the  basic  work 
unit  that  is  used  in  the  FDL  method  for  programming  the  Exploratory 
Development  effort  of  the  Laboratory.  Each  task  is  characterized 
primarily  in  terms  of  the  particular  aspect  of  technology  to  whose 
development  it  is  directed.  The  final  product  at  which  it  is  aimed 
is  a  fairly  well-defined  technology  objective. 

Progress  of  a  task  is  measured  by  the  responsible  task  engineer1 
assignment  of  a  value  to  a  descriptive  variable  designated  the  confi¬ 
dence  level.  For  a  particular  development  task,  n,  the  confidence 
level,  (CL)^,  may  assume  a  value  ranging  from  0.1  to  1.0  in  10  dis¬ 
crete  units.  In  this  range,  the  value  1.0  indicates  the  fulfillment 
of  the  task’s  technology  objective. 


From  another  viewpoint,  the  individual  development  task  is  also 
described  in  terms  of  the  funding  rates  and  the  amounts  of  in-house 
and  contract  engineers  that  should  be  devoted  to  it  to  achieve  differ 
ent  levels  of  (CL)^  within  different  time  frames. 

Each  development  task  is  related  specifically  to  each  proposed 
weapon  system  and  each  technical  goal.  This  relationship  is  depicted 
in  a  task  contribution  variable.  For  task  n,  b  is  assigned  a  value 
ranging  from  0.0  to  1.0  to  indicate  its  contribution  to  system  m. 

The  lower  end  of  the  scale  signifies  that  the  task  is  only  remotely 
associated  with  the  weapon  system;  the  higher  end  of  the  scale  signi¬ 
fies  that  it  is  absolutely  essential  to  completion  of  the  weapon 
system. 


Another  variable,  b^,  indicates  that  contribution  of  task  n  to 
technical  goal  k.  The  range  of  b  ^  extends  from  0.0  to  1.0  with 
lower  values  signifying  that  the  task  would  make  only  a  minor  con¬ 
tribution  to  technical  goal  k  and  the  higher  values  signifying  that 
the  task  might  contribute  a  potential  breakthrough  to  the  fulfillment 
of  the  goal. 
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f.  Costs.  As  indicated  in  the  above  description  of  the  devel¬ 
opment  tasks,  the  costs  tallied  for  each  task  are  the  funding  rates 
that  are  required  for  both  the  in-house  and  contract  work  that  would 
be  undertaken  at  different  levels  of  effort.  In  turn,  each  level  of 
effort  and  the  costs  that  it  entails  are  related  to  the  changes  that 
their  implementation  would  bring  about  in  the  confidence  level  of  the 
task  (the  extent  to  which  the  technology  objective  of  the  task  is  ful¬ 
filled)  in  different  time  frames.  These  relationships  are  displayed 
in  Fig.  4.  In  carrying  out  the  analysis,  three  funding  rates  are 
considered:  (1)  one-half  the  currently  scheduled  funding,  (2)  the 
currently  scheduled  funding,  and  (3)  double  the  currently  scheduled 
funding.  These  three  levels  are  illustrated  in  the  hypothetical  case 
shown  in  the  figure. 
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FISCAL  YEARS 

(  Show  progress  made  by  the  end  of  each  FY  for  each  resource  level ) 


(  CL  )  Measures  progress  toward  fulfillment  of  Task  technology  objective 

0  Projected  progress  with  funding  one-half  current  schedule 

Projected  progress  with  funding  on  current  schedule 

Projected  progress  with  funding  double  current  schedule 

•  Indicates  the  year  in  which  technology  objective  is  required 
for  System  m . 


FIGURE  4.  FDL  Method  Confidence  Level  Display  Chart 
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g.  Timing.  The  FDL  method  employs  a  timing  concept  that  is 
similar  to  that  employed  in  TORQUE  to  credit  a  task  with  achieving 
its  objective  in  a  time  frame  suitable  for  specific  future  weapon 
systems  or  technical  goals.  This  timeliness  function  is  shown  in 
Fig.  5.  It  is  based  on  the  progress  estimates  contained  in  the  con¬ 
fidence  level  relationships  and  the  dates  on  which  a  technology  is 
needed  for  incorporation  into  systems  or  fulfilling  technical  goals. 
From  the  confidence  level  relationship  is  taken  the  number  of  years 
that  would  be  required  under  given  funding  rates  to  achieve  a  (CL) 
of  0.8,  the  level  necessary  to  ensure  fit  into  the  system  or  goal. 

As  is  shown  in  Fig.  5,  the  value  assigned  to  f(t)  ,  the  timeliness 
variable  pertaining  to  task  n  with  regard  to  its  application  to  system 
m,  depends  upon  how  well  the  underlying  funding  rate  matches  the 
progress  of  the  task  to  the  time  that  it  is  needed. 


t 

nm 


The  number  of  years  required  to  achieve  a  (CL)  of  0.8 
in  Task  n  with  funding  rates  of  d, 

t  The  number  of  years  until  a  (CL)  of  0.8  in  Task  n  is 
nm  required  for  its  coupling  into  System  m. 


FIGURE  5.  FDL  Method  Timeliness  Function 


h.  Risk.  The  principal  element  of  risk  that  is  taken  into 
account  in  the  FDL  method  is  the  probability  with  which  the  output 
of  a  task  can  be  incorporated  into  a  weapon  system,  given  the  con¬ 
fidence  level  to  which  the  task  has  developed.  This  probability,  P  , 
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is  directly  related  to  the  confidence  level  values,  (CL)n,  as  shown 
by  the  right  vertical  scale  in  Fig.  4. 


4. •  Constraints 

Three  primary  types  of  constraints  are  incorporated  into  the  FDL 
method.  The  first  restricts  the  total  amount  of  funds  that  can  be  ex¬ 
pended  in  any  one  budget  year  of  the  program.  The  second  limits  the 
amount  of  in-house  engineering  manpower  that  can  be  used  in  the  sche¬ 
duled  tasks  for  the  next  year.  The  third  similarly  restricts  the 
amount  of  contract  engineering  manpower  that  can  be  used  in  the  same 
time  period. 

A  fourth  type  of  constraint  is  used  strictly  for  computation  pur¬ 
poses.  Although  it  is  the  main  source  of  the  large  number  of  in¬ 
equalities  in  the  resulting  computer  model,  it  in  no  way  has  anything 
to  do  with  the  substance  of  the  Exploratory  Development  allocation 
problem.  Because  a  number  of  discrete  resource  levels  may  alter¬ 
natively  be  allocated  to  any  one  task,  this  constraint  is  introduced 
for  each  task  to  prevent  the  computing  algorithm's  choosing  more  than 
one  resource  level  in  such  a  way  that  a  larger  amount  than  has  been 
engineered  is  recommended  for  a  task. 


5.  Decision  Algorithm 


The  factors  and  analytical  components  described  above  are  syn¬ 
thesized  in  the  FDL  method  to  develop  a  measure  of  value  that  is 
attributed  to  a  task  when  it  is  funded  at  a  specific  level. 
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( CL );nc^  =  the  change  in  (CL)^  over  the  next  year  due  to  the 
funding  rate  d 

(CL)^  =  the  level  of  (CL)^  at  the  beginning  of  the  program  year 

and  all  other  variables  are  as  defined  in  the  above  text. 

Total  value  of  the  Lab  program  of  Exploratory  Development  tasks 
is  accumulated  in  the  following  function. 

N  D 

Total  Value  =  V  Y  V  .X 

z-'  *-rf  nd  nd 
n  d 

where  X  ,  =  1,  if  task  n  is  funded  at  level  d. 
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The  decision  algorithm  consequently  consists  of  choosing  the 
is  such  a  way  to  maximize  the  total  value  subject  to  the  following 
constraints. 
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Available  contract  engineering  manpower, 

Available  in-house  engineering  manpower, 

Total  available  funds, 

1  (n  =  1,  2,  • • • >  N) 

amount  of  contract  engineering  manpower  required 
in  task  n  when  funded  at  level  d 

amount  of  in-house  engineering  manpower  required 
in  task  n  when  funded  at  level  d 

the  funds  that  will  be  expended  on  task  n  when 
funded  at  level  d. 
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In  the  original  formulation  of  the  FDL  method  a  linear  programming 
algorithm  was  used  as  the  search  and  decision  routine  for  the  selection 
of  the  tasks  to  be  included  in  the  Exploratory  Development  program 
(Ref.  14).  However,  because  that  was  judged  to  be  too  expensive,  a 
specialized  decision  rule  was  devised  and  coded  in  1968.  The  latter 
procedure  is  based  primarily  on  the  ratio  of  the  military  "utility'’ 
to  the  dollar  cost  for  each  of  the  alternative  resource  levels  speci¬ 
fied  beforehand  for  each  task  (Ref.  15). 

E.  CORNELL  AERONAUTICAL  LABORATORY  METHOD 

Under  contract  to  the  U.S.  Army  Materiel  Command,  the  Cornell 
Aeronautical  Laboratory  (CAL)  reported  in  1965  on  a  quantitative 
method  that  it  had  devised  to  guide  the  allocation  of  the  Exploratory 
Development  effort  (Ref.  16).  The  starting  point  of  this  method  is 
the  Army's  Qualitative  Materiel  Development  Objective  (QMDO),  describ¬ 
ing  the  general  characteristics  of  a  new  weapon  system.  Proposed 
QMDOs  are  decomposed  into  technologies  and  tasks,  for  each  of  which 
an  assessment  is  made  of  the  risk  of  success.  The  budget  available 
for  Exploratory  Development  is  allocated  among  the  tasks  in  a  manner 
that  attempts  to  take  into  account  the  importance  of  the  QMDOs  the 
tasks  support,  the  increase  in  the  chance  of  success  of  the  QMDO  if 
the  task  is  included  in  the  program,  and  the  cost  of  the  task. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  CAL  method  is  to  choose  the  set  of 
Exploratory  Development  tasks  that  has  the  maximum  expected  "value" 
to  the  Army  and  that  can  be  funded  within  the  budget  available  for 
the  next  program  year. 

2.  Control  Variables 

The  principal  control  variables  in  the  CAL  method  are  the  tasks 
that  are  to  be  undertaken  in  the  Exploratory  Development  program. 

While  control  focuses  first  at  the  level  of  the  development  tasks,  it 
ultimately  determines  the  weapons  that  will  be  developed  and  the 
particular  development  strategy  that  will  be  followed  in  the  program. 
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3 .  Factors  and  Their  Relationships 


a.  Value  Measure.  The  measure  of  value  that  is  to  be  maximized 
appears  to  be  a  scaled  unit  of  a  personal  or  subjective  indicator 
(possibly  averaged  over  a  number  of  subjects)  of  the  "essentiality" 

of  the  various  candidate  equipments  to  the  Army.  The  precise  meanings 
of  "value"  and  "essentiality"  are  not  completely  clear  in  the  report; 
however,  the  vagueness  of  these  concepts  is  recognized  and  their 
measurement  is  left  to  further  study. 

b.  Operational  Requirements.  In  the  formal  structure  of  the 
CAL  method,  no  direct  reference  is  made  to  the  operational  require¬ 
ments  of  the  Army. 

c.  Weapon  Systems.  The  overall  weapon  system  concept  incorpo¬ 
rated  into  the  CAL  method  is  the  QMDO  or  Qualitative  Materiel  Devel¬ 
opment  Objective.  The  QMDO  is  "a  statement  of  a  Department  of  the 
Army  military  need  for  developing  new  materiel,  the  feasibility  or 
specific  definition  of  which  cannot  be  determined  sufficiently  to 
permit  establishing  a  qualitative  requirement"  (Ref.  17). 

As  pointed  out  above,  little  has  been  done  in  the  development 
of  the  method  to  resolve  the  measures  of  "essentiality"  or  "value" 
that  are  to  be  applied  to  the  QMDOs.  This  was  left  to  later  work. 

As  the  method  is  formulated,  however,  the  QMDOs  are  proposals 
to  which  the  Exploratory  Development  effort  may  be  applied.  Insofar 
as  that  effort  is  not  limitless,  the  QMDOs  compete  with  each  other 
for  implementation  and,  in  that  sense,  are  substitutes  for  each  other 
even  though  they  may  not  all  fulfill  similar  functions. 

Each  overall  weapon  system  concept,  in  turn,  is  decomposed  into 
a  set  of  subsystems  or  materiel  concepts  which  describe 

...the  devices  or  items  envisioned  as  being  necessary 
to  meet  the  objectives  and  requirements  of  a  QMDO.  Asso¬ 
ciated  with  each  materiel  concept  is  a  set  of  critical  per¬ 
formance  parameters  which  are  quantified  expressions  of  the 
essential  factors  characterizing  the  performance  required  of 
the  materiel  concept.  All  materiel  concepts  for  a  given 
QMDO  are  considered  necessary  in  meeting  the  QMDO  (Ref.  16, 
p.  15) (emphasis  added). 
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In  other  words ,  all  of  the  materiel  concepts  in  a  QMDO  are  strict 
complements  to  each  other. 

.  d.  Technologies .  Each  subsystem  or  materiel  concept  in  the  CAL 
method  is  conceived  as  being  possibly  implemented  by  any  one  of  a 
number  of  composites  of  technologies.  Each  composite  is  formulated 
into  a  technical  approach  to  the  materiel  concept  which  details  the 
technological  advances  that  must  be  made  for  its  completion.  For 
any  single  materiel  concept,  there  is  a  menu  of  technical  approaches 
that  describes  the 

...proposed  alternative  methods  or  techniques  to 
satisfy  the  Materiel  Concept.  A  Technical  Approach  can 
have  one  or  more  Major  Barrier  Problem  Areas  associated 
with  it  which  must  be  overcome  by  accomplishing  proposed 
research  and  exploratory  development  tasks  in  order  to 
establish  the  feasibility  of  the  Technical  Approach  (Ref. 

16,  p.  15). 

Technical  approaches,  therefore,  can  be  considered  substitutable 
for  each  other  to  satisfy  a  materiel  concept;  in  this  case  they  are 
strict  substitutes,  the  completion  of  any  single  one  supporting  a 
materiel  concept  making  the  latter  feasible. 

e.  Development  Tasks.  The  specific  work  content  of  each  tech¬ 
nical  approach  is  spelled  out  in  terms  of  development  tasks,  time- 
phased  Research  and  Exploratory  Development  efforts. 

...Although  a  Task  may  be  further  subdivided  into 
Subtasks,  the  Task  is  considered  the  lowest  element  of 
the  QMDO  Model  Structure  since  it  is  the  element  which 
is  funded  as  well  as  the  one  which  furnishes  the  basic 
data  for  QMDO  planning  analyses  and  synthesis.  Associ¬ 
ated  with  each  Task  is  one  or  more  technical  goals  which 
represent  the  objective  of  the  Task  (Ref.  16,  p.  16). 

Each  task  is  characterized  also  by  the  pattern  of  resources  that 
is  used  in  its  execution.  These  resources  are  spelled  out  in  terms 
of  the  Army  Materiel  Command  R&D  Field  Establishments  involved  or  in 
terms  of  other  input  measures  considered  relevant  for  budgeting 
purposes . 
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The  development  task  structure  of  the  CAL  method  couples  Explora¬ 
tory  Development  efforts  specifically  with  individual  materiel  or 
system  concepts.  It  preserves  the  identification  of  the  tasks  and 
their  technology  objectives  with  the  QMDOs  they  support. 

The  tasks  making  up  a  technical  approach  may  need  to  be  ordered 
according  to  some  sequence  where  the  outputs  of  one  task  may  be  the 
foundation  of  another  task  required  for  progress  toward  the  feasi¬ 
bility  of  the  technical  approach.  However,  where  the  sequence  may 
be  chosen  on  the  basis  of  other  considerations,  the  CAL  method  pro¬ 
poses  that  the  tasks  be  ordered  according  to  decreasing  values  of  the 

■pi  cl/ 

ratio,  CQS-p  where  risk  is  the  complement  to  the  task  probability  of 
success  (described  below). 

Through  the  formulation  of  the  tasks,  some  consideration  is  also 
given  to  the  alternative  development  strategies  that  might  be  followed 
to  achieve  technical  goals  and  materiel  concepts.  The  resolution  of 
the  development  strategy  is  a  consequence  of  including  among  the  eli¬ 
gible  alternative  actions  parallel  technical  approaches  and  backup 
tasks  for  the  fulfillment  of  any  single  technical  approach. 

f.  Costs.  For  each  task  or  technical  approach,  costs  are 
tallied  in  two  different  ways:  (1)  the  total  funds  that  will  be  nec¬ 
essary  to  support  the  particular  task  in  the  next  budget  year  and  (2) 
the  profile  of  the  different  resources  that  will  be  used  to  carry  out 
the  task  work  during  the  same  period. 

The  fund  requirements  are  the  total  money  costs  that  will  be  in¬ 
curred  in  the  next  budget  year  to  carry  out  the  work  planned  under  any 
single  task.  This  quantity  is  the  sum  of  all  costs  associated  with  a 
task  regardless  of  where  or  on  what  they  are  spent  during  the  budget 
period. 

The  CAL  method  also  makes  provision  for  tallying  individual 
resources  that  will  be  employed  in  a  particular  task  throughout  the 
budget  year.  In  this  case,  each  U.S.  Army  Materiel  Command  R&D  Field 
Establishment  is  treated  as  a  separate  resource  and  the  extent  that 
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any  one  of  these  might  be  employed  to  work  on  a  task  is  accounted  for 
separately. 

g.  Timing.  The  time -phasing  of  the  various  efforts  included 
within  any  given  task  is  the  principal  avenue  through  which  direct 
consideration  is  given  to  the  timing  of  the  Exploratory  Development 
program  in  the  CAL  method.  Less  directly,  the  specification  of  the 
weapon  systems  to  be  included  among  those  eligible  for  development 
over  the  next  budget  year  must  also  contain,  at  least  implicitly, 
some  consideration  of  the  timing  of  their  being  made  available. 

h.  Risk.  The  primary  objective  of  the  CAL  method  is  to  choose 
the  set  of  tasks  so  as  to  maximize  the  expected  "value TT  of  the  Explora¬ 
tory  Development  program  to  the  Army. 

It  is  the  expected  value  in  the  sense  that  the  method  takes  into 
account  explicitly  the  estimated  probability  of  success  of  each  task 
in  the  Exploratory  Development  work  aimed  at  demonstrating  the  feasi¬ 
bility  of  the  QMDO. 

The  probability  of  success,  and  consequently  the  expected  value, 
of  a  particular  QMDO  is  derived  from  the  individual  and  compounded 
success  probabilities  of  the  various  operations  in  the  development 
process.  The  basic  building  block  for  this  evaluation  is  the  proba¬ 
bility  of  success  assigned  the  individual  tasks.  For  a  particular 
technical  approach,  the  probability  of  success  is  the  probability  that 
all  the  tasks  necessary  to  the  completion  of  the  approach  will  be 
successful.  The  success  probability  for  a  materiel  concept  is  the 
probability  that  at  least  one  of  the  technical  approaches  will  suc¬ 
ceed.  In  turn,  the  probability  of  success  of  a  QMDO  is  the  proba¬ 
bility  that  all  of  the  materiel  concepts  making  up  the  QMDO  will  be 
jointly  successful. 

The  expected  value  of  a  QMDO  is,  therefore,  the  product  of  the 
"essentiality"  measure  and  the  probability  of  success  of  the  QMDO. 
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4.  Constraints 


Three  types  of  constraints  are  considered  in  the  CAL  method: 

QMDO  contraints,  resource  constraints,  and  the  budget  constraint. 

Provision  is  made  in  the  structure  of  the  model  for  the  analyst 
to  specify  whether  a  QMDO  must  be  in  the  final  program  or  whether  its 
inclusion  can  be  elected.  If  a  QMDO  must  be  in  the  program,  a  QMDO 
constraint  is  devised  to  express  the  condition. 

If  prior  policy  determinations  have  been  made  to  support  the 
various  Materiel  Command  R6D  Field  Establishments  at  no  less  than 
some  minimum  levels  of  effort,  resource  constraints  are  used  in  the 
analytical  framework  to  ensure  that  support.  Basically,  such  a  con¬ 
straint  tallies  the  amount  expended  at  a  particular  field  establish¬ 
ment  by  all  of  the  tasks  included  in  the  program  and  prevents  the  sum 
from  totaling  less  than  the  chosen  floor  level. 

The  budget  constraint  ensures  that  the  total  costs  incurred  to 
fulfill  the  chosen  tasks  will  not  exceed  the  funds  available  for  the 
Exploratory  Development  program. 

5.  Decision  Algorithm 

The  factors  that  are  taken  into  account  in  the  CAL  method  are 
structured  for  deciding  the  budget  allocation  in  a  fashion  that  can  be 
represented  by  the  diagram  contained  in  Table  4. 

To  choose  the  program  of  technical  approaches  (and  consequently 
the  first- year  tasks)  to  be  undertaken  in  Exploratory  Development 
during  the  next  budget  year,  the  CAL  method  uses  a  rather  simple  pro¬ 
cedure.  However,  it  does  involve  extensive  computations. 

This  procedure  begins  with  the  program  that  includes  all  of  the 
technical  approaches  of  all  the  QMDOs.  It  subsequently  eliminates 
individual  technical  approaches  on  the  basis  of  increasing  values  of 
the  ratio,  (change  in  expected  program  value) /(change  in  total  program 
costs),  associated  with  the  elimination  of  individual  technical  ap¬ 
proaches.  Individual  technical  approaches  contribute  to  the  expected 
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value  of  the  program  to  the  extent  they  contribute  to  the  proba¬ 
bility  of  success  of  the  related  QMDO.  Also,  if  the  elimination  of  a 
technical  approach  reduces  to  zero  the  probability  of  success  of  a 
materiel  concept,  the  remaining  value  of  the  supported  QMDO  is  elimi¬ 
nated. 


TABLE  4.  CAL  METHOD  FRAMEWORK  OF  FACTORS  TAKEN  INTO  ACCOUNT 
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Eliminations  are  carried  out  in  order  of  increasing  values  of  the 
ratio  as  long  as  they  do  not  involve  violation  of  a  QMDO  constraint  or 
a  resource  constraint.  Those  eliminations  that  would  violate  one  of 
the  constraints  are  skipped. 

The  procedure  is  continued  until  the  total  costs  of  the  resulting 
program  are  reduced  to  the  funding  available  for  Exploratory  Develop¬ 
ment,  that  is,  the  total  costs  satisfy  the  budget  constraint. 
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F.  HERCULES  METHOD 


The  planning  staff  of  the  Hercules  Corporation  has  formulated  a 
quantitative  method  to  guide  the  allocation  of  its  Independent  Research 
and  Development  (IR&D)  funding  among  proposed  development  projects. 
These  projects  are  quite  similar  in  substance  to  military  Exploratory 
Development  work.  In  this  method,  the  products  that  are  generated 
by  the  development  projects  are  analyzed  in  terms  of  their  marketa¬ 
bility,  technological  content,  the  development  work  remaining  to  be 
done,  the  cost  of  that  development  work,  and  the  risks  involved  with 
each  of  these  features.  Available  funds  are  allocated  to  the  devel¬ 
opment  of  the  products  that  are  expected  to  earn  the  most  profits  for 
the  company. 

To  the  extent  that  could  be  determined,  the  allocation  deter¬ 
mined  by  this  method  appears  to  act  as  the  basic  budget  for  company 
IR&D  work.  Exceptions  can  be  taken  with  the  allocation  and  changes 
made  in  it  on  the  basis  of  considerations  that  could  not  be  incorpo¬ 
rated  into  its  formal  operation. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  Hercules  method  is  to  choose  the  set 
of  development  tasks  that  can  be  financed  with  the  company rs  IR&D  funds 
and  that  can  be  expected  to  generate  the  maximum  profits  for  the 
company. 

2.  Control  Variables 

The  principal  control  variable  in  this  method  is  the  amount  of 
funds  that  should  be  spent  on  specific  development  tasks  within  each 
technology. 

3.  Factors  and  Their  Relationships 

a.  Value  Measures.  In  the  Hercules  method,  the  value  of  the 
IR&D  effort  is  measured  in  terms  of  the  dollar  sales  and  profits  it 
will  generate  for  the  company.  This  value  measure  is  fairly  readily 
quantified.  Although  predicted  profits  from  the  development  work  may 
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not  be  estimated  easily,  they  correspond  to  accounting  data  that  are 
regularly  measured  and  recorded. 

b.  Operational  Requirements.  In  the  structure  of  the  Hercules 
method,  the  concept  that  corresponds  most  closely  to  military  opera¬ 
tional  requirements  is  the  set  of  characteristics  that  the  company's 
marketing  organization  attributes  to  each  market  in  which  the  company 
wants  to  participate.  In  addition,  the  marketing  organization  identi¬ 
fies  specific  subsets  of  these  characteristics  to  help  delineate  the 
features  that  should  be  incorporated  in  any  company  product  for  that 
market. 

c.  Weapon  Systems.  Two  concepts  in  the  Hercules  method  corre¬ 
spond  to  military  weapon  systems  in  the  structure  of  the  general  model: 
(1)  actual  equipment  and  other  products  that  embody  the  output  of  the 
development  effort  and  (2)  potential  contract  research  support  work 
for  which  the  company  might  qualify  as  a  result  of  the  particular 
project.  In  both  cases,  the  marketing  organization  must  also  relate 
these  products  to  the  markets  in  which  they  can  be  sold  by  estimating 
the  potential  sales  volume  of  the  company  and  its  profit  margin  in 
these  markets. 

d.  Technologies.  Using  the  marketing  estimates,  the  engineer¬ 
ing  staff  determines  the  technologies  that  will  be  necessary  to  satisfy 
the  mission  requirements  of  the  project's  products.  In  the  technology 
composition,  two  kinds  of  relationships  are  admitted  among  the  differ¬ 
ent  technologies.  First,  the  engineers  establish  the  independent  tech¬ 
nology  requirements  of  the  project.  Second,  within  any  single  tech¬ 
nology  requirement,  the  engineers  may  identify  a  number  of  technologies 
that  alternatively  can  fulfill  the  requirement.  In  this  second  kind  of 
relationship,  the  alternative  technologies  are  treated  as  perfect 
substitutes  for  each  other.  In  the  first  relationship,  the  product 

is  pres.umed  to  be  composed  of  strictly  complementary  technology  re¬ 
quirements. 

e.  Development  Tasks.  Besides  setting  out  the  technological 
content  of  the  project,  the  engineering  organization  estimates  the 
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nature  of  the  development  work  that  must  be  completed  in  each  tech¬ 
nical  area  to  satisfy  the  project.  Specifying  these  basically  resolves 
into  delineating  the  development  tasks  that  must  be  performed  and  the 
technical  goals  of  those  tasks. 

The  company 's  research  organization  reviews  the  development  tasks 
to  estimate  the  feasibility  of  the  technical  goals.  It  also  is  charged 
with  the  responsibility  to  identify  any  other  company -funded  work  or 
work  being  done  outside  the  company  that  can  be  drawn  upon  to  facili¬ 
tate  the  development  work  that  must  be  done. 

f.  Costs.  The  company Ts  research  organization  also  has  the  re¬ 
sponsibility  to  estimate  the  funding  that  will  be  required  to  fulfill 
the  technical  goals  of  each  development  task  in  a  technology.  For  each 
task  a  series  of  funding  estimates  is  made,  the  different  funding  levels 
corresponding  to  different  probabilities  of  success  that  the  goal  will 
be  achieved  within  the  specified  time  allowance. 

g.  Timing.  The  Hercules  method  incorporates  into  its  allocation 
procedure  consideration  of  the  timing  of  the  development  work  both  in 
terms  of  the  marketing  of  its  products  and  in  terms  of  the  time  needed 
to  achieve  a  technical  goal.  In  estimating  the  sales  volume  and 
profitability  of  any  product  resulting  from  a  development  project,  the 
marketing  organization  must  estimate  the  date  by  which  the  introduc¬ 
tion  of  the  product  must  be  made  to  capitalize  on  the  market  available 
for  it.  This  date,  in  turn,  establishes  when  the  technological  devel¬ 
opment  work  must  be  completed  to  permit  successful  innovation  of  the 
product. 

On  the  other  hand,  the  research  organization  must  estimate  the 
relationship  between  the  pattern  of  funding  that  might  be  devoted  to 
a  specific  development  task  and  the  time  that  would  be  required  to 
achieve  a  reasonable  level  of  the  technical  goal. 

h.  Risk.  The  sources  of  the  riskiness  of  a  development  project 
are  introduced  at  several  points  in  the  structure  of  the  Hercules 
method.  The  marketing  organization's  estimates  of  the  sales  volume 
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and  profitability  of  a  project’s  product  are  made  in  terms  of  the 
probabilities  with  which  the  specific  levels  are  expected  to  occur. 

In  other  words,  the  marketing  organization  must  specify  the  proba¬ 
bilities  that  it  believes  applies  to  the  actual  outcome  of  alternate 
sales  volumes  and  profit  margins  for  a  particular  product  when  it  is 
marketed. 

In  specifying  the  technological  composition  of  the  product  of  a 
development  project,  the  engineering  organization  is  also  expected  to 
estimate  the  probability  that  new  research  must  be  done  in  each  tech¬ 
nology  to  satisfy  the  project. 

Last,  the  research  organization  must  set  the  probability  that  the 
technical  goal  of  a  development  task  will  be  achieved  within  the 
allowed  time  under  any  given  funding  pattern  for  the  task. 

4.  Constraints 

The  only  constraint  that  is  imposed  in  the  method  is  that  the 
program  of  development  tasks  must  be  devised  within  the  given  budget 
of  IR&D  funds. 

5.  Decision  Algorithm 

All  of  the  analytical  features  of  the  method,  for  all  the  pro¬ 
posed  development  projects,  are  integrated  into  a  single  mathematical 
function  that  is  used  for  computing  the  allocation  of  the  IR&D  funds. 


2  (WV)j  =  Z  (RF)_.(PS)j(PF)j(PG)j(PC)j 


where  (WV)  . 
(RF). 


=  the  weighted  value  of  project  j, 

=  the  probability  that  project  j  will  be  ready  by  (YP).., 
the  necessary  completion  date,  given  the  technology 
composition  of  the  project  and  the  development  require¬ 
ments  of  the  technologies.  In  turn,  (RF)^  is  a  function 
of  (PR)..,  the  probability  that  the  research  on  tech¬ 
nology  i  for  project  j  will  be  completed  by  (YP)^. 
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(PS).. 

(PF) j 

(PG) j 
(PC). 


(PR)^^  is  a  function  of  X^,  the  dollar  expenditure  on 
technology  i. 

=  the  total  sales  volume  of  the  project  output, 

=  the  company  margin  on  project  j, 

=  the  probability  that  the  market  will  develop  as  expected, 

=  the  probability  that  the  company  can  capture  some  share 
of  the  expected  market. 


The  algorithm  used  in  the  method  is  basically  a  routine  for 
searching  the  number  of  alternative  allocations  of  funds  among  the 
technologies  to  find  the  one  with  the  maximum  expected  profit  payoff. 
The  procedure  follows  an  approximation  method  whereby  a  shift  of  funds 
from  one  technology  to  another  is  tried.  The  effects  of  such  a  shift 
are  traced  through  the  probabilities  of  the  technology  requirements 
being  fulfilled  to  the  expected  profit  values  of  the  relevant  projects. 


G.  ARMY  MISSILE  PLAN 

The  Army  Missile  Plan  is  generated  by  the  Future  Missile  Systems 
Division  of  the  Army  Missile  Command.  The  procedure  for  organizing 
the  plan  has  been  under  development  for  a  few  years  and  has  reached  a 
point  at  which  implementation  becomes  feasible. 

Figure  6  shows  the  general  flow  of  the  materials  prepared  in  the 
development  of  the  plan.  Unlike  the  methods  described  above,  the 
Army  Missile  Plan  does  not  employ  a  formal  mathematical  optimization 
procedure  to  determine  the  allocation  of  development  resources  among 
the  technologies.  It  consists  basically  of  setting  out  systematically 
and  visibly  the  technological  improvements  and  resources  that  would 
support  projected  weapons  requirements.  In  this  way,  the  plan  pro¬ 
vides  information  on  a  continuing  basis  for  formulating  Missile  Com¬ 
mand  programs  in  Research,  Exploratory  Development,  and  Advanced 
Development. 
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WEAPON 

REQUIREMENTS 


USER 

REQUIREMENT 

PRIORITY 


Primary  Objective  of  the  Method 


The  primary  objective  of  the  Army  Missile  Plan  is  to  show  the 
time-phased  set  of  development  tasks  and  resources  that  would  support 
weapons  concepts  proposed  for  future  deployment  within  a  specific 
scheme  of  priorities. 


2 .  Control  Variables 

Because  the  plan  is  largely  an  information  system  without  pro¬ 
vision  for  some  kind  of  optimization  procedure,  it  is  not  clearly 
structured  from  the  point  of  view  of  a  decision  maker  who  has  dis¬ 
cretion  over  a  primary  set  of  control  variables.  However,  the  general 
weapons  concepts  and  the  funding  requirements  of  the  technological 
tasks  play  pivotal  roles  in  the  planning  procedure. 


Factors  and  Their  Relationships 


a.  Value  Measures.  The  Missile  Plan  does  not  utilize  a  simple 
single  measure  of  value  to  depict  how  well  a  particular  course  of 
action  under  the  plan  would  achieve  its  primary  objective.  Of  course, 
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its  primary  objective  is  not  set  out  in  a  fashion  that  its  satisfac¬ 
tion  can  be  readily  measured  by  a  cardinal  metric. 

The  plan  procedure  does  use,  however,  a  set  of  concepts  to  assign 
priorities  to  the  weapon  systems  that  the  development  work  is  to  sup¬ 
port.  It  also  uses  criteria  to  assign  ordinal  ratings  to  the  specific 
technological  tasks  that  would  be  performed  in  the  developmental 
effort.  Both  the  priorities  and  ratings  contribute  to  the  establish¬ 
ment  of  task  rankings  that  serve  a  purpose  similar  to  other  value 
measures. 


b.  Operational  Requirements.  Army  operational  requirements  are 
introduced  into  the  planning  procedure  through  a  number  of  documents 
generated  by  the  Combat  Development  Command.  These  include  the 
Qualitative  Materiel  Development  Objectives  (QMDO),  Qualitative 
Materiel  Requirement  (QMR),  Advanced  Development  Objective  (ADO), 
Proposed  Qualitative  Materiel  Development  Objectives  (PQMD),  Proposed 
Qualitative  Materiel  Requirements  (PQMR),  and  Draft  Proposed  Qualitative 
Materiel  Development  Objectives  (DPQMDO).  Each  document  describes 
fairly  generally  the  military  need  for  some  new  item,  system  or 
assemblage  that  will  aid  the  Army  in  fulfilling  its  various  missions. 
These  user  requirements  are  derived  from  Army  plans  that  indicate 

the  time  period  in  which  it  would  be  desirable  to  introduce  the  new 
item  into  the  force  structure.  Fifteen  such  requirements  were  used 
to  derive  the  1969  Missile  Plan. 

c.  Weapon  Systems.  On  the  basis  of  the  operational  requirements, 
the  Missile  Command  Advanced  Systems  Laboratory  and/or  industry  de¬ 
veloped  a  number  of  weapon  system  concepts.  For  the  1969  plan,  70 
weapon  systems  were  proposed  to  fulfill  the  15  statements  of  require¬ 
ments. 

The  relative  priority  of  each  of  these  weapon  systems  is  set  in 
terms  of  the  target  date  on  which  the  system  should  begin  its  Contract 
Definition  Phase  (CDP).  This  date  is  established  on  the  basis  of 
three  factors:  (1)  the  user's  priority  for  the  system  as  evidenced 
in  the  Combat  Development  Command  Management  Information  System,  (2) 
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the  timing  of  re -buys  or  planned  phase-outs  for  the  existing  weapon 
system  that  currently  fills  the  general  role,  and  (3)  the  possible 
timing  of  any  new  technological  advance  that  should  be  available  for 
the  CDP  of  the  weapon  system. 

d.  Technologies.  The  technical  laboratories  of  the  Missile 
Command  and  the  Munitions  Command  determine  the  technological  compo¬ 
sition  of  weapon  system  and  subsystem  concepts.  They  then  go  on  to 
estimate  the  current  state  of  the  relevant  technologies  and  the  ad¬ 
vance  that  would  be  necessary  to  fulfill  the  concept  designs. 

Besides  the  operational  requirements  that  give  rise  to  specific 
weapons  concepts,  some  user  requirements  are  stated  in  terms  of  general 
functional  capability  that  might  be  introduced  into  current  or  future 
weapons  or  used  to  support  them.  The  technological  components  of 
these  requirements  are  also  established  at  this  stage  of  the  planning 
procedure. 

'  e.  Development  Tasks.  In  the  process  of  estimating  the  tech¬ 
nological  advance  that  would  be  necessary  to  fulfill  the  designs  of 
the  proposed  weapons,  the  technical  laboratories  also  prepare  a  plan 
of  research  tasks  to  achieve  the  technological  advance.  These  tasks 
are  planned  in  terms  of  the  number  of  years  that  will  be  necessary  to 
achieve  the  technical  advance  involved,  the  stage  of  development  into 
which  the  work  will  fall  in  each  year,  and  the  schedule  of  resources 
needed. 

Ratings  are  assigned  to  these  tasks.  A  rating  of  I  indicates 
that  the  task  is  critical  to  the  fulfillment  of  the  weapon  concept 
objectives.  A  rating  of  II  is  assigned  to  tasks  that  have  a  high 
potential  of  technical  advance  but  are  not  absolutely  essential  to 
reaching  the  weapon  concept  objectives.  Marginal  tasks  having  both 
a  low  potential  and  contributing  little  to  specific  weapon  concepts 
are  assigned  a  rating  of  III. 

f.  Costs.  At  the  same  time  that  they  prepare  the  plan  of  re¬ 
search  tasks,  the  laboratories  also  estimate  their  funding  and  manpower 
requirements. 
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g.  Timing.  Each  research  task  and  the  corresponding  requirements 
for  funding  and  manpower  are  planned  with  timing  patterns  that  will 
satisfy  the  original  target  date  set  for  the  contract  definition  phase 
of  the  specific  (or  earliest)  weapon  system  they  are  to  support.  The 
target  date  for  the  contract  definition  phase  is  set  in  the  first 
instance  for  each  weapon  system  on  the  basis  of  the  priority  of  the 
weapon  and  the  "critical ff  development  task  planned  by  the  labs.  This 
task  is  absolutely  essential  to  the  development  of  the  weapon,  and  the 
estimate  of  the  time  required  to  achieve  the  associated  technical  ad¬ 
vance  sets  a  limit  on  how  soon  contract  definition  can  progress.  The 
timing  of  all  the  other  tasks  linked  to  a  weapon  system  is  then  re¬ 
arranged  to  dovetail  with  the  target  contract  definition  phase  date. 

h.  Risk.  There  is  no  provision  in  the  Army  Missile  Plan  for 
assessing  and  taking  into  account  the  chance  of  success  of  the  indi¬ 
vidual  tasks  or  the  probability  with  which  the  necessary  technological 
advances  will  be  made.  The  tasks,  their  resource  requirements,  and 
their  timing  are  set  out  as  though  they  will  all  turn  out  as  planned. 

4.  Constraints 

Inasmuch  as  the  plan  sets  out  the  technological  development  work 
that  should  be  done  to  support  new  weapons  concepts  derived  from 
independent  statements  of  operational  requirements,  it  does  not  incor¬ 
porate  explicitly  any  resource  or  political  constraints  on  that  devel¬ 
opment  work.  However,  when  the  plan  is  subsequently  used  in  the  pro¬ 
gramming  process  to  determine  the  allocation  of  Exploratory  Develop¬ 
ment  funds,  the  total  budget  available  is  treated  as  an  external 
constraint.  .  ■ 

5.  Decision  Algorithm 

The  development  tasks  worked  out  in  the  planning  process  are 
grouped  according  to  common  technological  features  and  arrayed  on  the 
basis  of  the  ordinal  ratings  assigned  them  in  the  labs  and  the  pri¬ 
orities  of  the  weapons  in  which  they  will  be  used.  The  recommended 
budget  allocation  is  then  derived  by  distributing  the  available  funds 
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down  the  array,  beginning  with  the  highest  rated  work,  until  the  funds 
are  exhausted.  The  lowest  rated  work  supporting  weapons  concepts 
farther  in  the  future  or  marginally  useful  to  the  planned  weapons  is 
the  last  to  be  funded  and  is  postponed  if  the  higher  rated  work  absorbs 
all  the  funds.  No  formal  mathematical  optimization  procedure  is  used. 

H.  AIR  FORCE  DOL  PLAN 

In  1969,  the  Air  Force  Systems  Command  Directorate  of  Laboratories 
(DOL)  began  to  develop  a  method  that  it  hopes  will  be  used  by  all  Air 
Force  Laboratories  to  plan  their  programs  in  the  Exploratory  Develop¬ 
ment  and  advanced  development  categories.  The  resulting  method  is 
less  quantitative  than  the  others;  however,  it  attempts  to  link  ex¬ 
plicitly  the  development  plan  to  the  Air  Force  systems  concepts  that 
have  been  projected  for  the  future.  The  general  outline  of  the 
development  of  the  plan  is  contained  in  Fig.  7. 

The  Air  Force  Flight  Dynamics  Laboratory,  which  attempted  the 
experimental  application  of  TORQUE  to  its  program  in  1968,  is  one  of 
the  labs  within  this  Directorate. 

I.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  DOL  method  is  to  devise  laboratory 
programs  in  Exploratory  Development  and  Advanced  Development  that 
support  the  weapons  systems  projected  for  future .deployment  by  the 
Air  Force. 

2 .  Control  Variables 

Inasmuch  as  the  DOL  method  does  not  attempt  to  allocate  the  devel¬ 
opment  budget  with  precision,  it  does  not  have  control  variables  as 
such.  However,  the  analysis  of  the  plan  does  recommend  a  set  of  de¬ 
velopment  projects  and  tasks  that  should  be  undertaken  to  support  the 
future  systems.  Consequently,  the  recommendations  of  tasks  serve  the 
same  purpose  as  control  variables. 

3.  Factors  and  Their  Relationships 

a.  Value  Measures.  The  DOL  method  does  not  explicitly  use  any 
measure  of  value  or  scheme  of  priorities. 
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FIGURE  7.  Air  Force  DOL  Plan 

b.  Operational  Requirements .  For  present  planning  purposes ,  DOL 
uses  13  desired  capabilities  in  the  same  capacity  as  operational  re¬ 
quirements.  Desired  capabilities  are  incorporated  into  the  Technology 
Planning  Guide  as  descriptions  of  the  capabilties  that  are  necessary 
to  accomplish  the  missions  and  submissions  assigned  to  the  Air  Force. 
The  evolution  of  each  capability  is  spelled  out  for  a  number  of  years 
into  the  future  by  five-year  intervals,  each  interval  depicting  a 
step  change. 

As  shown  in  Fig.  7,  these  operational  requirements  are  derived 
from  more  basic  statements  of  national  security  objectives. 

Interestingly,  the  13  desired  capabilities  that  are  used  are 
the  same  as  those  used  in  the  experimental  application  of  TORQUE 
(cf.  Fig.  11,  in  Chapter  IV). 


c.  Weapon  Systems.  For  each  increment  in  an  operational  require¬ 
ment,  the  Mr  Force  Systems  Command  identifies  a  number  of  system 
concept  possibilities  and  the  most  probable  systems;  AFSC  also  breaks 
out  the  system  characteristics  and  pertinent  performance  data  as  well 
as  critical  subsystems  and  subsystem  requirements.  To  carry  out  this 
analysis,  AFSC  consults  with  the  various  operating  commands  that  would 
deploy  these  systems. 

d.  Technologies .  Along  with  the  AFSC,  the  different  technical 
laboratories  specify  the  technical  goals  that  compose  the  subsystems 
of  the  projected  weapons.  For  example,  they  would  specify  the  desired 
specific  fuel  consumption  (SFC),  thrust /weight,  thrust,  etc.,  for  the 
propulsion  of  a  large  subsonic  cruise  aircraft.  From  these  goals,  the 
labs  identify  the  gaps  in  technology  that  must  be  filled.  Similar 
technical  goals  are  compiled  into  Technical  Planning  Objectives  (TPOs) 
for  each  lab. 

The  development  work  that  must  be  done  to  support  a  particular 
future  system  can  be  displayed  in  terms  of  the  TPOs.  In  turn,  the 
TPOs  serve  as  focal  points  for  the  specific  elements  of  work.  Each 
Technical  Planning  Objective  details  the  approaches  that  will  be  taken 
to  fulfill  the  technical  goal  and  the  milestones  that  management  can 
use  to  measure  progress  in  the  work. 

e.  Development  Tasks.  Each  Technical  Planning  Objective  in  a 
laboratory  consists  of  a  number  of  basic  work  units  called  technical 
efforts.  For  organizational  and  planning  purposes,  these  technical 
efforts  are  grouped  into  tasks  according  to  the  commonality  of  their 
technical  goals.  Tasks,  in  turn,  are  also  grouped  into  projects. 
Finally,  projects  in  similar  technical  areas  are  combined  into  the 
program  elements  that  make  up  the  Exploratory  and  Advanced  Develop¬ 
ment  program  categories. 

f.  Costs .  In  describing  the  technical  efforts  that  would  be 
undertaken  in  a  particular  Technical  Planning  Objective,  the  labora¬ 
tories  must  also  incorporate  a  resource  summary  estimating  the  fund¬ 
ing  requirements  of  the  technical  effort. 
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g.  Timing.  The  time  pattern  in  the  evolution  of  the  desired 
capabilities  is  the  principal  avenue  by  which  the  timing  of  the  devel¬ 
opment  program  is  established.  This  time  pattern  displays  discrete 
steps  in  the  projected  capabilities  by  five-year  intervals.  These 
changes  drive  the  new  systems  that  must  be  deployed  in  the  different 
time  frames  and,  consequently,  the  time  pattern  of  development  that 
must  be  accomplished  to  support  those  systems. 

h.  Risk.  The  DOL  method  makes  no  provision  for  the  technical, 
cost,  or  timing  risks  that  may  characterize  individual  technical 
efforts  or  combinations  of  those  efforts. 

4.  Constraints 

The  DOL  method  does  not  explicitly  consider  any  special  constraints 
that  should  be  applied  to  the  pattern  of  technical  efforts  it  might 
recommend. 

5.  Decision  Algorithm 

Inasmuch  as  the  DOL  method  does  not  attempt  to  allocate  the  re¬ 
sources  available  for  implementing  the  development  program,  it  does  not 
use  a  set  of  rules  for  choosing  a  preferred  set  of  technical  efforts. 

I.  ARMY  RESEARCH  PLAN 

The  Army  Research  Office  publishes  the  Army  Research  Plan  bi¬ 
ennially.  This  plan  evaluates  the  balance  of  the  Army’s  Research 
and  Exploratory  Development  programs  and  provides  general  guidance 
to  the  field  agencies  on  the  shifts  that  might  be  made  in  their 
programs .  The  plan  does  not  attempt  to  devise  precise  allocations 
of  the  related  program  budgets.  It  is  essentially  a  management 
information  system  based  on  case  studies  that  summarize  important 
factors  for  each  research  and  technology  area.  From  these  factors, 
it  derives  its  recommendations  on'  shifts  in  program  emphasis. 

1.  Primary  Objective  of  the  Method 

The  primary  objective  of  the  Army  Research  Plan  is  to  recommend 
shifts  in  the  Research  and  Exploratory  Development  programs  that 
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respond  to  the  long-range  concepts ,  operational  requirements ,  materiel 
and  nonmateriel  objectives  projected  for  the  Army. 

2 .  Control  Variables 

Although  the  Army  Research  Plan  does  not  attempt  to  control  the 
programs  precisely,  it  does  recommend  shifts  in  the  funding  patterns 
among  the  elements  and  subelements  of  the  program.  Consequently, 
it  is  formulated  in  a  way  that  conceives  funding  of  the  program  ele¬ 
ments  as  the  principal  instrument  of  control. 

3.  Factors  and  Their  Relationships 

a.  Value  Measures.  The  Army  Research  Plan  employs  three  types 
of  value  measures.  First,  it  uses  a  scheme  of  priorities  that  ex¬ 
presses  the  relative  importance  of  the  operational  requirements. 
Second,  it  uses  a  subjective  measure  of  the  contribution  a  technology 
makes  to  an  operational  requirement.  Third,  it  also  uses  a  measure 
based  on  the  analyst’s  judgment  regarding  the  timely  progress  of  a 
development  effort  toward  its  technical  goal  under  current  funding 
conditions. 

b.  Operational  Requirements.  The  Army  Research  Plan  procedure 
derives  operational  requirements  from  three  sources : 

1.  The  Combat  Development  Objectives  Guide  (CDOG)  contains 
56  Operational  Capabilities  Objectives*  approved  by  the 
Department  of  the  Army. 

2.  The  CDC  Army  85  Concept  Study  lists  a  set  of  operational 
requirements . 


An  Operational  Capability  Objective  is  defined  as  "A  Department 
of  the  Army  approved  description  of  an  operational  capability 
desirable  of  achievement  in  a  specified  time  frame  10  or  more 
years  in  the  future.  An  0C0  is  responsive  to  envisioned  future 
operational  concepts,  within  constraints  of  probable  technological 
capabilities.  OCOs  taken  together  provide  a  comprehensive  goal  to 
planners  of  doctrine,  organization,  tactics,  logistical  support 
and  development,  and  provide  guidance  for  research  and,  together 
with  QMDO,  for  exploratory  development.”  (Ref.  17  ,  p .  B-2) 
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3.  The  Army  Strategic  Plan  describes  a  number  of  objectives 
that  provide  guidance  for  materiel  and  nonmateriel  develop¬ 
ment  efforts. 

Each  of  these  documents  also  assigns  measures  of  the  relative  pri¬ 
orities  that  it  recommends  be  given  to  its  requirements. 

c.  Weapon  Systems.  The  plan  does  not  treat  weapon  systems  ex¬ 
plicitly  or  at  all  link  them  directly  to  its  analysis.  However,  one 
of  the  major  references  used  for  guidance  in  the  development  of  the 
plan  is  the  Army  Long-Range  Technological  Forecast.  Volume  III  of 
that  document  describes  advanced  system  concepts,  which  include  ex¬ 
amples  of  the  materiel  systems  that  would  be  possible  if  certain  tech¬ 
nological  capabilities  were  achieved  and  applied. 

d.  Technologies.  The  scientific  and  technical  areas  considered 
in  the  Army  Research  Plan  are  described  in  the  Army  Long-Range 
Technological  Forecast.  The  Forecast  attempts  to  extrapolate  for  a 
period  of  about  20  years  the  scientific  and  technological  capabilities 
that  are  particularly  relevant  to  Army  missions  and  needs.  It  is 
oriented  primarily  to  the  physical  sciences.  Social  and  behavioral 
science  research  subjects  are  contained  in  the  Department  of  the  Army 
Behavioral  and  Social  Science  Plan  for  Military  Operations  in  the 
Developing  Nations. 

The  Army  Plan  appraises  the  ability  of  the  ongoing  Research  and 
Exploratory  Development  programs  to  support  the  operational  require¬ 
ments  in  a  two-step  procedure.  First,  it  uses  a  matrix  to  show  the 
support  that  various  efforts  in  the  Research  program  provide  the 
efforts  in  the  Exploratory  Development  program  (Fig.  8).  A  second 
matrix  describes  the  relationships  of  the  efforts  of  the  Exploratory 
Development  program  to  the  operational  requirements.  For  these  pur¬ 
poses,  the  12  program  elements  in  the  Research  program  category  are 
the  Research  efforts;  the  30  program  elements  in  the  Exploratory 
Development  program  category  are  the  Exploratory  Development  efforts. 

The  project  officers  in  the  Office  of  the  Chief  of  Research  and 
Development  make  an  entry  in  each  cell  of  these  two  matrices  to 
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describe  the  technical  relationship  between  the  technical  fields  or 
between  the  technical  field  and  the  operational  requirement.  In  the 
first  matrix,  this  entry  depicts  the  potential  relevance  of  the  re¬ 
search  element  to  the  Exploratory  Development  element.  In  the  second 
matrix,  the  entry  depicts  the  potential  relevance  of  the  Exploratory 
Development  element  to  the  operational  requirement. 


EXPLORATORY  DEVELOPMENT  ELEMENTS 


OPERATIONAL  REQUIREMENTS 


UJ 


FIGURE  8.  Matrices  in  the  Army  Research  Plan 

For  both  matrices,  five  levels  of  relevance  are  distinguished: 

Blank  =  no  application  perceived 

1  =  minor  application 

2  =  substantial  application 

3  =  major  application 

4  =  essential  application 
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The  matrix  entries  are  based  upon  the  subjective  judgment  of  the 
project  officers  and  represent  an  abstraction  of  complex  considera¬ 
tions  . 

The  project  officers  also  write  qualitative  appraisals  that 
elaborate  on  the  content  of  the  different  projects  and  provide  a 
rationale  to  support  the  matrix  entries. 

e.  Development  Tasks.  For  the  Exploratory  Development  program, 
OCRD  has  designed  a  more  detailed  matrix  relating  the  projects  within 
each  program  element  to  the  56  operational  requirements.  The  relevance 
entries  in  this  matrix  are  averaged  to  derive  the  relevance  entries 
relating  the  program  elements  to  the  operational  requirements.  At 

the  level  of  greater  detail,  the.  project  officers  also  take  into 
account  the  accomplishments  of  R&D  activities  external  to  the  Army 
to  judge  the  relevance  of  the  technical  area  to  the  requirement. 

f.  Costs .  In  the  plan,  the  past  and  projected  funding  trends 
of  the  individual  Research  and  Exploratory  Development  elements  are 
thoroughly  documented.  On  the  basis  of  this  information  and  a  number 
of  other  considerations,  the  project  officers  make  another  entry  in 
each  cell  of  the  matrices.  This  entry  reflects  their  judgment  on  the 
adequacy  of  support  being  directed  to  the  program  element  as  it  per¬ 
tains  to  the  particular  application.  A  fivefold  numerical  classifi¬ 
cation  scheme  is  used  to  describe  the  adequacy  of  the  current  level 
of  funding: 

1  -  significantly  underfunded 

2  -  less  than  optimal  funding 

3  -  optimal  funding 

4  -  greater  than  optimal  funding 

5  -  significantly  overfunded 

In  this  case,  optimal  funding  is  the  level  of  support  that  would 
provide  a  reasonable  probability  of  achieving  the  particular  technical 
goal  on  time. 


72 


Other  factors  taken  into  consideration  in  the  appraisal  of  the 
adequacy  of  currently  planned  funding  include: 

a.  The  sensitivity  of  the  rate  of  advance  in  the  technical  area 
to  funding  changes 

b.  The  rate  of  advance  in  the  technical  area  at  planned  funding 
relative  to  the  advance  needed 

c.  The  relative  umarginal  utility"  of  increased  funding 

d.  The  adequacy  of  efforts  outside  the  Army  to  meet  needs  in 
the  technical  area 

e.  The  proximity  of  the  "threshold  level"  in  the  technical  area 

f.  The  proximity  of  the  point  at  which  needed  "in-house"  capa¬ 
bility  will  be  lost 

g.  The  availability  of  personnel,  facilities,  and  equipment 

h.  The  quantity  of  valid  "unfunded  requirements" 

i.  The  number  of  acceptable  but  unfunded  proposals. 

As  in  the  appraisals  of  the  technical  relationships,  the  project 
officers  provide  the  rationale  for  their  judgments  on  the  matrix 
entries  in  qualitative  written  analyses. 

g.  Timing.  The  Army  Research  Plan  takes  into  account  operational 
requirements  for  a  period  up  to  20  years  into  the  future.  Technologi¬ 
cal  advances  needed  to  fulfill  the  operational  requirements  are  em¬ 
bodied  in  the  technical  projects,  the  timing  of  which  is  obviously 
driven  by  the  time  horizon  of  the  relevant  requirement.  From  the  nec¬ 
essary  dovetailing  of  projects,  the  project  officers  must  determine 
whether  the  time  pattern  of  funding  for  any  project  is  suitable.  For 
the  purposes  of  the  plan,  primary  emphasis  is  given  to  those  efforts 
that  must  be  carried  out  within  the  next  one  to  five  years. 

h.  Risk.  At  most,  the  Army  Research  Plan  considers  qualitatively 
the  risk  involved  in  the  different  development  projects.  In  setting 
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out  the  relationship  of  a  technical  area  to  an  operational  require¬ 
ment,  relevance  is  defined  as  "potential"  relevance.  This  qualifica¬ 
tion  is  intended  to  recognize  the  risk  involved  in  actually  achieving 
the  project  goal  as  well  as  the  uncertainty  regarding  the  precise 
nature  of  the  impact  the  technical  advance  would  have  on  the  opera¬ 
tional  requirement. 

In  the  appraisal  of  the  adequacy  of  the  currently  planned  funding 
for  a  project,  the  criterion  of  "optimal"  funding  contains  a  considera¬ 
tion  of  risk.  Optimal  funding  is  the  level  that  provides  a  reasonable 
probability  of  achieving  the  project  goal  on  time. 

4.  Constraints 

A  number  of  constraints  are  introduced  into  the  deliberations  on 
the  shifts  that  should  be  made  in  the  pattern  of  Research  and  Explora¬ 
tory  Development  undertaken.  At  least  implicit  consideration  is  given 
to  maintaining  some  level  of  in-house  capability  in  the  different 
technical  areas.  Also,  the  plan  is  concerned  with  the  total  budget 
that  is  available  to  do  development  work  within  its  time  frame.  Under 
the  procedures  of  the  plan,  consideration  must  also  be  given  to  (a) 
the  need  for  the  Army  to  be  generally  involved  in  Research  and  Explora¬ 
tory  Development ,  (b)  R&D  activities  being  carried  on  outside  the  Army, 
(c)  development  work  directed  by  the  OSD  and  the  JCS,  and  (d)  special 
Army  responsibilities  in  the  field  as  executive  agent  for  the  Military 
Services . 

5.  Decision  Algorithm 

The  Army  Research  Plan  does  not  employ  a  set  of  precise  mathe¬ 
matical  rules  to  arrive  at  its  recommendations  on  the  emphasis  and 
direction  that  should  be  given  the  Research  and  Exploratory  Develop¬ 
ment  programs.  A  small  staff  planning  group  interprets  the  appraisals 
and  funding  information  provided  by  the  project  officers.  The  group 
analyzes  this  information,  taking  into  account  several  other  factors, 
some  of  which  have  been  described  above.  Among  these  other  factors 
are  the  priorities  given  the  various  operational  requirements  and  the 
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constraints  discussed  in  more  detail  in  the  preceding  sections.  In 
addition,  the  group  incorporates  the  appraisals  and  recommendations 
in  the  Army  Research  Council  Report,  the  recommendations  of  the  earlier 
Army  Research  Plan,  and  the  several  responses  made  to  those  recommenda¬ 
tions. 

Finally,  the  staff  breaks  the  program  elements  into  three  priority 
groups.  The  first  group  includes  those  efforts  the  staff  evaluates 
to  be  most  in  need  of  increased  funding.  The  second  group  includes 
the  efforts  that  also  need  increased  funds  but  with  a  lower  priority 
than  the  first  group.  The  third  group  consists  of  the  efforts  whose 
funding  is  considered  satisfactory. 

Final  review  and  approval  of  the  group’s  conclusions  and  recom¬ 
mendations  are  made  by  the  directors  of  the  OCRD  directorates. 

J.  ANOTHER  SERVICE  METHOD 

One  of  the  Services  has  experimented  with  another  specially  for¬ 
mulated  quantitative  method  to  devise  the  mix  of  projects  that  should 
be  included  in  its  Exploratory  Development  program.  This  method  at¬ 
tempts  to  quantify  the  considerations  given  a  number  of  factors  that 
bear  upon  the  allocation  problem  and  to  integrate  them  into  a  single 
index  that  the  method  proposes  be  used  to  guide  the  allocation  of  the 
available  budget. 

Experience  with  the  method  was  short;  the  Service  terminated  its 
application  after  one  year. 

1.  Primary  Objective  of  the  Method 

The  designers  of  the  method  have  not  addressed  directly  the  princi¬ 
pal  purpose  for  which  the  method  has  been  formulated.  However,  one 
might  infer  that  the  primary  objective  of  the  method  is  to  devise  a 
development  program  that  best  serves  the  pattern  of  operational  require¬ 
ments  composing  the  overall  mission  of  the  Service. 
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2.  Control  Variables 


The  method  uses  as  the  principal  instrument  of  control  the  amount 
of  funds  that  should  be  allocated  to  individual  development  projects. 

3.  Factors  and  Their  Relationships 

a.  Value  Measures.  The  method  employs  subjectively  determined 
weighting  factors  to  measure  the  relative  importance  of  the  different 
operating  requirements  and  the  contribution  of  each  technological 
advance  to  the  operating  requirements. 

b.  Operating  Requirements.  A  set  of  29  of  the  Serviced  exist¬ 
ing  general  operating  requirements  were  used  in  the  method.  Senior 
staff  personnel  assigned  a  weight,  ranging  from  1  to  10,  to  each  re¬ 
quirement,  reflecting  its  relative  importance  in  the  overall  mission 
of  the  Service. 

c.  Weapon  Systems.  The  procedure  of  the  method  does  not  require 
reference  to  a  set  of  weapon  systems  that  might  be  deployed  in  the 
future. 


d.  Technologies.  The  method  groups  the  development  work  to  be 
done  into  general  technological  areas.  These  areas,  in  turn,  are  re¬ 
lated  to  the  individual  operating  requirements. 

e.  Development  Tasks.  The  method  basically  analyzes  development 
tasks  such  as  those  reported  on  Form  DD  1498.  In  the  particular  appli¬ 
cation  750  such  tasks  were  considered.  Each  task  sets  out  its  techni¬ 
cal  objective.  Those  tasks  with  objectives  that  fit  together  in  some 
way  are  combined  into  a  project. 

In  the  procedure  of  the  method,  each  task  is  assigned  a  number  of 
weights,  each  of  which  measures  the  impact  of  the  task  on  a  particular 
operating  requirement.  These  weights  range  from  0.1  to  1.0,  using 
interpolations  of  the  following  scale : 

1.0  -  critical  to  developing  the  requirement 

0.7  -  extends  the  current  ability  to  fulfill  requirement 

0.3  -  improves  the  current  ability  to  fulfill  requirement 
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In  this  analysis  of  the  tasks,  the  method  also  determines  the 
number  of  concurrent  approaches  that  are  being  taken  to  the  same  tech¬ 
nical  objective. 

f.  Costs.  The  cost  data  used  in  the  method  consists  of  estimates 
of  the  funding  that  will  be  required  to  complete  each  project. 

g.  Timing.  The  timing  of  the  technical  work  and  the  required 
funding  is  derived  from  the  dates  by  which  the  operational  requirements 
should  be  fulfilled.  In  the  particular  application,  operational  require¬ 
ments  to  1982  were  considered.  Technical  objectives,  development  tasks, 
and  funding  time  patterns  were  worked  out  to  meet  those  requirements. 

h.  Risk.  This  method  does  consider  the  probabilities  that  the 
different  technical  objectives  will  be  successfully  achieved.  Those 
probabilities  are  based  on  the  probability  of  success  of  the  indivi¬ 
dual  development  tasks.  For  an  individual  task,  the  probability  of 
success  is  the  probability  that  the  effort  will  achieve  the  desired 
technological  advance  within  the  given  time  frame  under  the  currently 
projected  funding.  Achievement  of  a  particular  technical  objective 
requires  only  the  success  of  at  least  one  of  the  tasks  undertaken  in 
parallel  toward  that  objective. 

4.  Constraints 

The  method  employs  two  types  of  constraints  on  the  patterns  of 
development  tasks  that  can  be  considered.  The  first  constraint  re¬ 
quires  that  some  minimum  level  of  in-house  capability  be  maintained 
in  particular  technical  areas.  The  second  requires  including  among 
the  funded  tasks  those  that  are  specially  sponsored  or  directed  from 
outside  the  immediate  program  management. 

5.  Decision  Algorithm 

The  method  uses  four  steps  in  its  algorithm  to  determine  the 
preferred  ordering  of  funding  for  -the  individual  development  tasks. 

First,  it  calculates  the  total  value,  V . ,  of  the  technical  objec¬ 
tive  j. 
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V.  =  £  w.c.  . 
3  t  ii] 


where  W_^  -  measure  of  relative  importance  of  operational  requirement  i, 

and  c^  .  =  contribution  of  technical  objective  j  to  operational 
requirement  i. 

Second,  it  calculates  the  expected  value,  EV  . ,  of  technology 
objective  j. 


EV.  =  V.P. 
3  3  3 


where  =  the  probability  of  success  of  technical  objective  j , 

p  =  i  -  n  a-?  )  , 

3  k=l  K 

P^  =  probability  of  success  of  individual  approach  k, 

and  n  =  number  of  approaches  taken  in  parallel  to  technical 
objective  j. 

Third,  the  method  calculates  the  desirability  index,  D ^ ,  of 
technical  objective  j. 

D.  =  EV./F. 

3  3  3 

where  F.  =  funding  required  for  completion  of  technical  objective  j. 

Finally,  the  method  orders  the  tasks  according  to  descending 
values  of  their  desirability  indexes. 

In  the  particular  application,  the  allocation  (actually,  distri¬ 
bution  of  a  budget  cut)  was  accomplished  by  dividing  the  tasks  into 
three  equal  groups.  The  top  third  was  funded  at  the  requested  level, 
the  middle  third  was  cut  the  average  cut  required,  and  the  bottom 
third  was  cut  twice  the  average  cut  required. 
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IV.  CRITIQUE  OF  QUANTITATIVE  METHODS 


In  this  chapter,  the  methods  for  allocating  development  resources 
are  analyzed  within  the  framework  of  the  General  Model  set  out  in 
Chapter  II.  Each  major  feature  of  the  models  (primary  objective, 
control-variables,  factors,  constraints,  and  decision  algorithm)  is 
discussed  in  turn. 

This  analysis  has  a  number  of  purposes.  First,  it  provides  a 
setting  for  describing  desirable  characteristics  of  the  formal  quan¬ 
titative  features  and  planning  procedures  that  should  be  employed  in 
a  methodology  for  allocating  the  resources.  Second,  it  attempts  to 
ascertain  the  internal  consistency  of  the  methods  that  have  been  re¬ 
viewed  in  detail.  Would  the  resulting  allocations  achieve  the  goals 
that  the  formulators  of  the  methods  have  set  for  them?  Third,  it 
points  out  the  extent  to  which  the  features  of  the  methods  contain  or 
violate  desirable  characteristics  of  any  methodology  applied  to  the 
problem  of  allocating  resources  within  Exploratory  Development.  Would 
the  allocation  generated  by  the  method  produce  desirable  results  in 
the  Exploratory  Development  program?  Fourth,  in  the  process  of  work¬ 
ing  at  the  above  purposes,  it  attempts  to  assemble  some  evidence  on 
the  feasibility  and  advisability  of  using  a  particular  quantitative 
method  to  determine  the  allocation  of  the  Exploratory  Development 
budget.  At  the  extremes,  what  might  appear  to  be  good  characteristics 
of  a  quantitative  method  may  not  be  feasible  or  may  be  terribly  com¬ 
plicated  and  costly.  On  the  other  hand,  what  might  be  feasible  may 
not  be  desirable.  That  is,  there  might  be  little  confidence  that 
following  the  allocation  generated  by  a  particular  explicit  method 
may  ultimately  contribute  to  higher  order  defense  goals . 
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Rigid  criteria  cannot  be  established  for  such  a  critique.  Within 
the  framework  of  the  General  Model,  the  development  of  some  features 
of  a  specific  quantitative  method  may  be  a  matter  of  choice.  In  a 
particular  instance,  the  analyst  may  be  able  to  choose  any  one  from  a 
set  of  options  to  devise  a  method  without  prejudicing  its  quality. 
However,  any  specific  choice  may  need  to  be  consistent  with  choices 
dealing  with  other  features  of  the  method,  and  its  development  may 
need  to  satisfy  a  number  of  conditions  to  ensure  good  quality  analysis. 

The  precise  formulation  of  the  method  will  depend  upon  (1)  the 
special  interests  of  the  analyst  or  manager,  (2)  the  kinds  of  informa¬ 
tion  available,  (3)  the  technical  training  of  the  analysts  and  man¬ 
agers,  and  (4)  the  costs  incurred  to  compose  and  implement  the  method. 

All  of  the  allocation  methods  that  were  reviewed  are  quantitative 
to  some  extent.  That  is,  they  all  use  numerical  expressions  for  at 
least  some  of  the  factors  that  they  try  to  consider  systematically. 
However,  the  extent  to  which  they  rely  upon  numerical  measures  of  fac¬ 
tors  and  apply  formal  mathematical  procedures  to  those  measures  varies 
significantly  among  the  methods.  Frequently,  for  purposes  of  the 
analysis,  the  methods  fall  into  two  fairly  consistent  groups.  One  of 
these  is  referred  to  as  the  "more  quantitative"  methods.  They  generally 
assign  numerical  measures  to  all  the  factors  that  they  consider  to  be 
important  in  the  allocation  problem  and  proceed  to  derive  a  single 
measure  of  merit  for  each  proposed  development  project.  These  methods 
also  formally  calculate  the  precise  allocation  of  resources  that  should 
be  made  to  each  project.  The  group  consists  of  the  Industrial  Analog, 
TORQUE,  NOL,  FDL,  CAL,  Hercules,  and  Another  Service  methods. 

The  other  group  is  referred  to  as  the  "less  quantitative"  methods. 
These  methods  generally  do  not  try  to  assign  numerical  measures  to  some 
important  factors.  They  leave  the  consideration  of  these  factors  pri¬ 
marily  to  the  judgment  of  the  decision  maker  who  determines  the  alloca¬ 
tion  of  resources  among  the  proposed  projects  without  formal,  optimizing 
calculations.  This  group  consists  of  the  Army  Missile  Plan,  the  DOL 
Plan,  and  the  Army  Research  Plan. 
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A.  PRIMARY  OBJECTIVE  OF  A  QUANTITATIVE  METHOD 
1.  Discussion 


The  primary  objective  specified  for  the  quantitative  method  is 
extremely  critical  to  the  resolution  of  the  resource  allocation  prob¬ 
lem.  If  the  objective  is  misstated  or  inappropriate,  the  programs 
generated  by  the  method  will,  most  likely,  not  promote  the  intended 
end  results . 

Because  of  the  limited  perspective  of  the  DoD  Exploratory  Devel¬ 
opment  planning  context,  the  objective  devised  expressly  for  a 
quantitative  method  applied  to  it  may  not  be  a  direct  measure  of  DoD’s 
national  security  goals.  However,  the  objective  must  be  properly 
devised  to  reflect  the  role  delegated  to  Exploratory  Development  so 
that  the  allocations  generated  by  the  method  will  contribute  to  the 
broader  Defense  responsibility  for  national  security.  In  other  words, 
effective  Exploratory  Development  planning,  outside  a  completely  com¬ 
prehensive  plan  of  all  Defense-related  activities,  first  requires 
sufficient  delegation  of  responsibility  and  effective  subdivision  of 
objectives  and  incentives. 

Insofar  as  several  approaches  may  be  taken  to  Exploratory  Develop¬ 
ment  planning,  with  equally  satisfactory  results,  the  objective  of  a 
quantitative  allocation  method  may  not  be  unique.  That  is,  depending 
upon  the  approach  taken  to  the  planning  problem,  an  appropriate  objec¬ 
tive  might  be  tailored  according  to  this  limited  phase  of  Defense 
planning . 

If  the  allocations  generated  by  the  quantitative  method  are  to 
support  the  development  of  weapon  systems,  the  primary  objective  of 
the  method  must  include  some  consideration  of  how  the  demand  for 
progress  in  a  technology  is  derived  from  the  coupling  of  Exploratory 


81 


Development  into  the  overall  weapon  development  process.*  If  the 
magnitude  of  this  derived  demand  could  be  quantified  readily,  it  would 
measure  the  "value”  of  the  progress  in  the  technology  as  a  component 
of  one  or  several  systems. 

However,  some  formulations  of  the  primary  objective  can  circum¬ 
vent  to  some  extent  the  need  for  a  complete  solution  to  the  rather 
difficult  derived  value  problem.  The  scope  of  the  analytical  approach 
for  allocating  resources  within  Exploratory  Development  must  be  suffi¬ 
ciently  limited  if  the  problem  is  to  remain  at  all  manageable.  There¬ 
fore,  adequate  precautions  must  be  taken  to  ensure  that  the  quantita¬ 
tive  method  used  is  consistent  with  the  overall  Defense  objective  to 
provide  national  security  efficiently.  Such  precautions  can  take  the 


In  contrast  with  the  systems  support  approach  to  Exploratory 
Development  planning,  another  approach  might  be  taken  using  the 
accomplishments  within  each  technology  area  as  the  express  ob¬ 
jectives  of  the  planning  method.  One  variation  on  this  approach 
would  be  to  develop  a  minimum  cost  program  of  Exploratory  Devel¬ 
opment  work  to  achieve  specified  values  for  the  "pacing  parame¬ 
ters"  in  each  technology  area.  Similarly,  with  a  fixed  budget, 
the  objective  could  be  to  devise  a  program  that  will  maximize  the 
value  of  the  pacing  parameter  in  a  particular  technology  area 
while  achieving  specified  values  of  those  in  all  the  other  tech¬ 
nologies.  Further,  the  pacing  parameters  can  be  set  at  different 
values  in  a  series  of  solutions  to  the  method  to  trace  out  the 
marginal  resource  and  "opportunity  costs"  of  shifts  in  the  profile 
of  pacing  parameter  progress . 

Generally,  the  pacing  parameter  requirements  are  developed, 
either  explicitly  or  implicitly,  from  broader  studies  of  systems. 
However,  if  a  technology-oriented  approach  were  adopted  and  a 
directly  derived  relationship  between  technology  and  Defense 
demands  were  not  required,  the  pacing  parameters  might  well  become 
uncoupled  from  systems  projections  and  be  treated  as  objectives 
in  themselves.  Because  such  an  uncoupling  could  lead  to  an 
expenditure  of  effort  in  areas  that  are  far  afield  from  Defense 
missions,  the  technology-oriented  approach  is  not  pursued  further 
in  this  report  (such  efforts  may  be  considered  exploitive). 
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form  of  explicit  (and  judicious)  assumptions  or  restrictions  on  the 
primary  objective  of  the  quantitative  method  reflecting  the  policies 
that  arise  in  the  broader  context.  For  example ,  with  fixed  mission 
goals  for  Defense  forces,  the  Exploratory  Development  objective  might 
be  to  minimize  the  cost  of  developing  the  technologies  of  new  weapon 
systems  that  fulfill  those  goals  and  achieve  some  minimum  level  of 
"cost  savings"  over  current  forces.  Or,  the  objective  might  be  to 
maximize  the  rate  of  return  to  "investment"  in  Exploratory  Develop¬ 
ment,  given  the  force  missions,  the  technology  configurations  of  new 
weapon  systems,  and  the  potential  "cost  savings"  of  alternative  new 
systems  over  current  forces . 

Another  approach  to  systems  support  planning  would  be  to  fix  the 
mission  goals,  possibly  at  current  levels,  and  the  budget  that  can  be 
expended  in  Exploratory  Development.  The  objective  then  could  be  to 
develop  those  technologies  that  are  components  of  new  weapon  systems 
maximizing  the  cost  savings  possible  over  current  forces. 

If  future' mission  goals  are  "increased"  over  those  that  hold  at 
present  in  such  a  way  that  no  conceivable  changes  in  the  current 
force  mix  or  force  levels  could  fulfill  the  goals,  a  somewhat  differ¬ 
ent  approach  is  necessary.  The  approach  would  still  be  basically  one 
in  which  a  comparison  of  costs  would  be  made  among  alternative  weapon 
systems  but,  in  this  case,  the  alternatives  would  be  all  the  proj ected 
weapon  systems  that  could  fulfill  the  goals . 

The  means  by  which  Exploratory  Development  planning  is  coupled 
to  the  overall  development- procurement-deployment  process  require 
much  to  be  determined  outside  the  Exploratory  Development  context. 
Mission  goals,  preferences  among  these,  and  the  savings  of  some  new 
systems  over  current  forces  or  other  new  systems  are  the  principal 
avenues  for  expressing  some  of  these  outside  factors  in  the  sugges¬ 
tions  made  above . 

Depending  upon  the  different  development  strategies  that  can  be 
followed,  the  costs  and  the  initial  operating  capability  dates  of  new 
weapon  systems  can  be  varied  significantly.  Consequently,  the  primary 
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objective  should  include  some  provision  for  evaluating  the  different 
time  phasings  that  are  possible.  Such  evaluation  should  make  explicit 
management’s  time  preferences  for  different  technology  progress  streams 
and  different  expenditure  streams . 

The  development  of  each  new  weapon  system,  the  initiation  of  work 
in  new  aspects  of  a  technology  area,  the  choice  of  funding  levels  and 
the  planning  of  project  completion  time  involve  some  risk  that  original 
intentions  will  not  be  precisely  fulfilled.  In  fact,  in  some  concep¬ 
tions,  the  principal  role  of  research  and  development  is  the  production 
of  the  knowledge  that  reduces  risk  to  a  range  sufficiently  narrow  to 
justify  procurement  and  deployment  decisions. 

Reduction  of  risk  by  more  intensive  or  parallel  efforts  obviously 
consumes  scarce  resources.  Consequently,  a  higher  chance  of  success 
in  some  technology  development  is  purchased  at  the  expense  of  the 
probability  of  success  or  the  level  of  progress  in  another  technology. 
Such  trade-offs  affect  directly  the  objectives  of  Exploratory  Develop¬ 
ment.  Therefore,  explicit  treatment  of  the  risk  conditions  and 
management’s  risk  preferences  in  the  formulation  of  the  objective 
would  provide  a  basis  for  evaluating  the  possible  substitutions  in 
these  dimensions. 

2 .  Primary  Objectives  of  the  Reviewed  Quantitative  Methods 

The  primary  objectives  specified  in  the  quantitative  methods  that 
have  been  reviewed  generally  fall  into  three  classes.  First,  there 
are  the  objectives  such  as  that  adopted  in  the  Hercules  method  that  do 
not  neatly  represent  the  goals  of  the  Defense  Exploratory  Development 
program.  Second,  some  objectives,  such  as  those  specified  for  the 
Industrial  Analog  and  the  Army  Missile  Plan,  are  quite  restricted  in 
scope,  depending  heavily  upon  explicit  consideration  of  decisions 
made  external  to  the  Exploratory  Development  planning  process.  Third, 
the  objectives  specified  in  many  of  the  reviewed  methods  are  quite 
broad  in  that  they  are  directed  at  generating  allocations  of  the 
budget  for  Exploratory  Development  that  will  maximize  some  measure 
of  overall  military  value. 
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The  particular  primary  objective  specified  in  the  Industrial 
Analog  (follow  the  patterns  of  technological  development  exhibited  by 
U.S.  industry)  is  hardly  appropriate  for  determining  the  allocation 
of  the  Defense  budget  for  Exploratory  Development. 

The  objective  of  research  and  development  effort  for  American 
industry  is  commerical  success .  Commercial  success  is  measured  prin¬ 
cipally  by  the  profitability  of  the  enterprise  but  other  indicators 
may  also  be  important  to  the  companies  involved  in  this  context,  such 
as  survival,  company  share  in  the  market  and  changes  in  that  share, 
sales  growth,  industry  leadership,  and  an  image  of  progressiveness. 

These  objectives  are  tenuously,  if  at  all,  related  to  the  national 
security  objectives  of  DoD,  Consequently,  the  connections  of  commer¬ 
cial  R&D  funding  practices  and  results  to  DoD  R&D  resource  allocation 
and  the  efficient  achievement  of  Defense  objectives  are  remote,  at 
best.  In  fact,  with  similar  conviction,  an  opposite  contention  might 
be  proposed  that  DoD  imitation  of  commercial  research  and  development 
funding  behavior  would  have  a  perverse  result  on  national  security 
obj  ectives . 

Actually,  there  need  be  little  concern  over  whether  the  company 
objectives  in  doing  commercial  research  and  development  are  consistent 
with  the  broader  economic  goal  that  its  activity  improves  the  overall 
performance  of  the  economy.  The  operation  of  the  markets  for  goods 
and  services  acts  as  an  effective  and  impersonal  test  for  the  broader 
goal.  Those  R&D  products  that  succeed  in  the  market  by  lowering  costs 
or  satisfying  a  new  range  of  demand  contribute  to  the  efficient  per¬ 
formance  of  the  economy.  Those  R&D  products  that  are  market  failures 
do  not  improve  the  economy’s  performance  and  do  not  impose  additional 
costs  on  the  economy  once  the  failure  is  phased  out.  The  parallel 
production  of  successful  products  that  contribute  to  the  economy’s 
performance  and  unsuccessful  products  that  absorb  scarce  resources 
while  being  developed  is  the  price  that  is  paid  for  a  continuous  flow 
of  ’’improvements”  in  a  context  of  large  uncertainties.  However,  once 
the  products  are  developed,  the  market  operation  generates  additional 
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clear  and  decisive  information  without  catastrophic  results  for  the 
economy . 

No  comparable  impersonal  test  exists  for  demonstrating  that 
products  of  Defense  research  and  development  will  contribute  to  the 
efficient  achievement  of  national  security  goals  if  included  in  the 
military  force  structure.  All  manner  of  physical  and  operational 
testing  can  be  devised  and  carried  out  on  old  and  new  weapon  systems 
alike.  However,  the  decisive  test  such  as  commercial  products  face 
when  placed  on  the  market  can  only  be  conducted  in  actual  war  condi¬ 
tions  when  repetitive  testing  of  alternative  weapon  systems  may  hardly 
be  possible  and  the  consequences  of  failure  may  be  a  national 
catastrophe . 

The  primary  objective  of  the  Army  Missile  Plan  is  properly  modest 
in  scope.  It  sets  out  to  generate  an  Exploratory  Development  program 
that  will  support  operational  requirements  and  weapons  concepts  that 
are  limited,  in  the  first  instance,  by  policies  determined  in  the 
larger  planning  context. 

On  the  surface,  at  least,  little  quarrel  can  be  made  with  the 
kind  of  primary  objectives  specified  for  the  TORQUE,  Naval  Ordnance 
Laboratory,  Air  Force  Flight  Dynamics  Laboratory,  and  Cornell  Aero¬ 
nautical  Laboratory  methods.  These  methods  attempt  to  generate  allo¬ 
cations  of  the  budget  for  Exploratory  Development  that  maximize  the 
"military  utility"  attainable  from  this  program.  However,  the  sugges¬ 
tion  of  such  an  objective  raises  immediately  two  suspicions.  First, 
how  does  the  analyst  go  about  measuring  such  heady  stuff  as  "military 
utility"?  This  anticipates  to  some  extent  the  analysis  below  of  the 
value  measures  that  are  employed  in  quantitative  allocation  methods. 
For  the  present,  it  is  sufficient  to  be  aware  that  special  care  must 
be  exercised  in  devising  this  measure  and  relating  it  to  the  Explora¬ 
tory  Development  activities  so  that  the  extent  to  which  they  fulfill 
the  primary  objective  is  clear.  Second,  how  can  one  be  certain  that 
in  fact  the  "military  utility"  indicated  for  a  particular  allocation 
by  the  quantitative  method  is  attainable?  In  applying  the  Hercules 
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method,  a  company  could  measure  the  profits  that  actually  resulted 
from  the  products  generated  in  the  recommended  development  program, 
without  resorting  to  the  data  used  in  the  method.  In  a  method  some¬ 
what  like  the  Army  Missile  Plan,  observations  can  be  made,  after  the 
recommended  development  budget  has  been  used,  on  whether  the  projected 
weapon  systems  were  actually  developed  and  on  the  actual  costs  incurred. 

In  most  of  these  methods,  the  factors  that  are  taken  into  account 
and  the  relationships  among  them  are  so  directly  tied  and  interwoven 
with  the  function  representing  the  primary  objective  that  they  are 
basically  inseparable.  However,  the  CAL  method  appears  to  be  struc¬ 
tured  so  that  it  can  be  readily  modified  to  accept  other  objectives, 
some  of  which  do  not  have  the  same  pitfalls  as  those  based  on  the 
"value"  or  "utility"  measures  utilized  in  the  TORQUE,  NOL  and  FDL 
methods.  For  example,  inasmuch  as  it  has  provision  for  considering 
the  risk  involved  in  the  outcome  of  development  work,  the  CAL  frame¬ 
work  could  be  used  to  investigate  the  maximum  probability  of  success 
that  might  be  planned  for  a  particular  QMDO,  given  a  fixed  budget  and 
specified  levels  of  success  probabilities  for  the  other  concepts .  In 
this  way,  the  analyst  might  generate  a  "menu"  of  Exploratory  Develop¬ 
ment  programs,  each  of  which  involves  the  same  level  of  total  expendi¬ 
ture  but  among  which  success  in  the  development  of  some  concepts  is 
traded  off  for  success  in  others.  Such  a  procedure  might  be  a  richer 
way  to  have  management  reveal  its  preferences  (or  "essentiality"  ■ 
measures )  for  the  various  QMDOs . 

Alternatively,  the  CAL  method  might  be  used  to  investigate  the 
minimum  cost  that  must  be  incurred  to  secure  a  certain  probability  of 
success  for  attaining  feasibility  of  a  specified  program  of  QMDOs. 

In  the  expression  of  the  primary  objective,  the  probability  of  success 
for  each  QMDO  might  be  equal  to  the  others }  or  a  preferred  probability 
might  be  specified  for  each.  Through  such  an  analysis,  an  evaluation 
might  be  made  of  the  costs  that  would  have  to  be  incurred  to  increase 
the  success  probabilities,  or  that  are  being  incurred  to  increase  the 
probability  of  success  at  the  margin  for  any  QMDO. 
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3 .  Conclusion 


Obviously,  the  precise  expression  of  the  primary  objective  of  a 
quantitative  method  for  allocating  the  Exploratory  Development  budget 
is  not  uncomplicated.  The  mechanisms  suggested  above  for  coupling 
the  primary  objective  of  the  method  to  the  overall  development- procure¬ 
ment-deployment  process  are  not  completely  satisfactory.  Besides  the 
formidable  quantification  problems  that  must  be  overcome  if  they  were 
to  be  incorporated  into  the  objective,  the  measures  used  to  express 
the  coupling  are  not  conceptually  precise.  For  example,  the  potential 
cost  savings  of  new  weapon  systems  over  current  forces  arise  at  any 
stage  in  the  development- procurement- deployment  process,  not  just  in 
the  Exploratory  Development  stage.  Each  stage  presents  its  own 
allocation  problem  that  must  also  be  resolved,  and  as  each  is  resolved, 
in  a  somewhat  circular  fashion  it  generates  information  that  is  useful 
to  the  other  stages. 

For  the  most  part,  the  primary  objectives  of  the  quantitative 
methods  reviewed  are  too  broad  in  scope.  While  their  intent  is  un¬ 
doubtedly  good  and  in  the  right  direction,  measurement  of  the  extent 
to  which  they  can  be  fulfilled  is  vague,  at  best,  and  their  verifica¬ 
tion  in  terms  of  actually  observed  results  that  can  be  readily  agreed 
upon  is  virtually  impossible. 

B .  CONTROL  VARIABLES 
1.  Discussion 

The  analyst  can  exercise  some  flexibility  in  the  way  he  specifies 
the  control  variables  within  the  formulation  of  the  method.  However, 
if  the  allocation  method  is  to  be  useful  for  structuring  the  Explora¬ 
tory  Development  program,  the  control  variables  used  in  the  model 
should  conform  fairly  close  to  management’s  conception  of  the  first- 
order  choices  it  must  make.  Obviously,  the  detail  incorporated  into 
the  method  may  be  somewhat  finer  than  might  generally  concern  the 
particular  level  of  management  involved.  While  the  process  is  some¬ 
what  cumbersome  and  not  symmetrical,  detail  can  be  aggregated  without 
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loss  of  significance.  On  the  other  hand,  if  the  relevant  tier  of 
management  is  concerned  primarily  with  the  development  of  technology, 
the  options  in  the  planning  method  should  not  be  formulated  in  a  way 
that  would  give  it  the  choice  of  systems  that  will  be  developed 
ultimately . 

Within  the  present  context  of  budgeting  Exploratory  Development, 
the  analyst  might  use  one  of  two  types  of  control  variables .  With  one 
type,  the  weapon  systems  themselves  must  be  chosen.  With  the  other, 
the  systems  are  fixed  in  a  general  way  and  their  precise  technical 
compositions  must  be  chosen. 

If  -the  systems  must  be  chosen,  they  become  the  principal  control 
variables  in  the  method.  They  are  chosen  to  fulfill  some  specified 
capability  requirements  or  missions  on  the  basis  of  the  objective 
adopted.  In  turn,  the  technology  developments  that  must  be  initiated 
to  support  these  systems  and  the  funds  that  must  be  budgeted  for  these 
technologies  follow  closely  from  the  choice  of  systems . 

In  the  other  approach,  the  general  weapon  systems  that  will  be 
developed  are  fixed?  but  the  specific  configurations  of  technologies 
composing  the  designs  of  the  systems  must  be  chosen.  In  this  case, 
the  principal  control  variables  are  the  technology  areas  in  which 
Exploratory  Development  should  be  undertaken.  These  choices  can  be 
stated  either  in  terms  of  some  critical  parameters  of  the  technology 
areas,  indicating  progress  adequate  for  the  proposed  weapon  systems, 
or  in  terms  of  the  funds  that  must  be  allocated  to  the  technology 
areas  to  ensure  the  required  progress.  In  any  event,  more  precise 
systems  designs  will  follow  closely  from  these  decisions,  and  the 
recommended  budgets  of  the  technology  areas  will  either  be  a  direct 
output  of  the  method  or  be  directly  derived  from  it. 

Besides  choosing  the  weapons  systems  or  technology  areas  in 
which  Exploratory  Development  work  will  be  done,  the  program  manager 
may  also  want  to  control  as  much  as  possible  the  calendar  time  that 
will  elapse  until  any  one  effort  is  completed.  By  adopting  different 
development  strategies  and  incurring  different  costs,  he  may  be  able 
to  change  the  time  between  the  inception  and  completion  of  a  project. 
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The  program  manager  may  want  to  be  able  to  control  to  the  extent 
possible  the  level  of  risk  that  he  must  bear  regarding  the  performance, 
cost,  or  timing  of  any  one  Exploratory  Development  undertaking. 

Provision  for  risk  is  an  explicit  recognition  that  the  predicted 
outcome  of  any  Exploratory  Development  effort  should  be  expressed  as  a 
probability  distribution.  The  manager fs  choice  of  a  risk  level  gives 
him  some  control  over  the  assurance  he  will  have  that  the  actual  out¬ 
come  will  approximate  the  intended  outcome.  Of  course,  he  may  not  be 
able  to  choose  the  level  of  risk  for  all  features  of  the  outcome 
simultaneously .  Greater  assurance  of  the  technology  performance 
characteristics  may  possibly  be  achieved  only  at  a  higher  expected 
cost,  greater  cost  risk,  or  both. 

2.  Control  Variables  of  the  Reviewed  Quantitative  Methods 

With  the  notable  exception  of  the  Cornell  Aeronautical  Laboratory 
method,  the  first-order  control  variables  of  the  methods  reviewed  are 
the  funds  that  are  to  be  devoted  to  individual  development  tasks  or  to 
technical  areas.  Of  course,  the  tasks  that  will  be  undertaken  and  the 
level  of  effort  expended  upon  them  follow  necessarily  from  that  choice 
of  the  distribution  of  funds.  The  CAL  method  reverses  that  order 
somewhat.  For  it,  the  first-order  control  variables  are  the  tasks 
that  will  be  performed.  These  tasks  are  performed  fully  or  not  at 
all  so  that  the  funds  allocated  to  any  one  cannot  vary  between  zero 
and  the  total  estimated  cost  of  it. 

These  control  variables  appear  to  be  the  most  relevant  for  the 
administration  of  the  Exploratory  Development  program  at  levels  of 
management  concerned  with  the  overall  distribution  of  the  budget. 

Also,  at  first  glance,  they  appear  to  satisfy  the  concern  expressed 
earlier  that  the  scope  of  Exploratory  Development  planning  should  be 
properly  delineated  to  reflect  its  somewhat  restricted  purview  of  the 
joint  problem  of  weapon  development,  procurement,  and  deployment. 

The  control  variables  in  the  methods  reviewed  have  at  least  two 
shortcomings.  First,  they  omit  some  very  important  considerations  that 
management  must  make.  Second,  they  are  frequently  specified  in  a  way 
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that  results  in  their  determining  some  fairly  remotely  related  aspects 
of  the  development  process  through  their  second-order  effects. 

Among  the  most  glaring  omissions  from  the  considerations  that 
management  must  make  is  the  discretion  it  can  exercise  over  the  timing 
and  the  risks  associated  with  the  technical  results  of  projects.  Most 
of  the  discussion  of  timing  and  risk  in  Exploratory  Development  is  con¬ 
solidated  below  in  sections  under  those  titles.  In  the  context  of  con¬ 
trol  variables,  however,  note  should  be  made  that  none  of  the  methods 
pay  much  attention  to  the  extent  to  which  management  can  control  the 
timing  of  a  task  and  the  technical  progress  a  task  can  achieve. 

The  second  shortcoming  is  particularly  evident  in  the  formulation 
of  TORQUE  and  the  methods  closely  patterned  after  it.  In  these  cases, 
the  primary  control  is  exercised  over  the  funds  allotted  to  the  various 
development  tasks.  However,  the  particular  configuration  of  tasks 
that  is  chosen  by  the  budget  allocation  algorithm  in  turn  determines 
the  weapons  that  will  be  developed,  the  operational  requirements  that 
will  be  satisfied,  and  the  missions  for  which  new  weapons  will  be 
developed.  This  is  undoubtedly  a  much  larger  responsibility  than 
anyone  would  want  to  vest  in  a  method  for  determining  the  allocation 
of  funds  for  Exploratory  Development. 

3 .  Conclusions 

Basically,  the  control  variables  specified  in  the  quantitative 
methods  reviewed  are  unobjectionable.  They  reflect  the  first-order 
concerns  and  choices  that  must  be  made  by  management  responsible  for 
allocating  the  Exploratory  Development  budget. 

The  principal  shortcomings  of  the  control  variables  used  result 
to  a  large  extent  from  faulty  formulation  of  the  relationships  of  such 
factors  as  timing,  risk,  technologies,  and  value  to  systems,  operational 
requirements,  and  primary  objectives.  These  are  discussed  more  fully 
below.  As  a  consequence,  the  discretion  exercised  over  the  control 
variables  of  the  allocation  of  Exploratory  Development  funds ,  in  many 
cases,  has  much  broader  implications  for  factors  that  should  fall 
outside  the  proper  range  of  the  matters  determined  in  Exploratory 
Development . 
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C.  FACTORS  AND  THEIR  RELATIONSHIPS 


The  functional  relationships  incorporated  into  the  allocation 
method  are  basically  abstract  expressions  of  the  real  conditions  that 
must  hold  among  the  important  factors  in  the  problem.  Ih  effect, 
the  factors  and  their  relationships  tally  the  questions  that  a  pro¬ 
posed  course  of  action  must  be  able  to  satisfy  to  ensure  that  it  is 
workable  and  that  there  is  some  reasonable  chance  for  the  intended 
results  to  actually  occur. 

From  another  point  of  view,  the  relationships  can  be  considered 
as  a  set  of  checks  on  the  connections  among  the  inputs  and  outputs 
of  the  different  stages  of  the  R&D  process.  The  purpose  of  these 
checks  is  to  shape  the  planning-decision  framework  so  that  it  rea¬ 
sonably  describes  and  is  consistent  with  the  real  counterpart  opera¬ 
tions  of  R&D  to  some  level  of  detail. 

As  the  critical  link  between  the  conceptual- analytical  framework 
for  planning  and  the  actual  occurrences,  the  relationships  among  the 
factors  are  an  important  determinant  of  the  usefulness  of  the  planning 
exercise.  Obviously,  if  the  functional  relationships  are  inaccurate 
descriptions  of  the  conditions  that  hold  among  the  factors,  the  re¬ 
sulting  plans  are  more  unlikely  to  bring  about  the  intended  develop¬ 
ment  outcomes.  Also,  if  some  important  conditions  that  hold  among 
the  factors  are  not  included  in  the  set  of  relationships,  the  programs 
planned  with  the  allocation  method  will  probably  generate  technology 
developments  at  variance  to  those  intended . 

Individual  relationships  among  the  factors  frequently  defy  ready 
classification  by  type  or  purpose.  All  of  them  generally  tie  together 
closely,  and  any  distinction  must  be  drawn  on  the  basis  of  different 
viewpoints  to  the  overall  problem  or  on  the  basis  of  the  different 
problem  features  formulated  in  the  set.  In  the  following  discussion, 
a  division  of  the  factors  and  their  relationships  is  used  to  emphasize 
the  different  problem  features  that  are  incorporated  into  the  model 
through  them. 
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1.  Value  Measures 


a.  Discussion .  In  the  present  context,  value  is  an  indicator 
of  the  extent  to  which  a  development  task  or  action  achieves  its 
objective.  The  relevant  relationships  involving  the  value  measure 
show  the  extent  to  which  different  levels  of  effort  in  the  task 
achieve  different  levels  of  the  objective.  Depending  upon  the  par¬ 
ticular  point  of  view  adopted,  the  results,  and  therefore  the  value, 
of  the  development  work  may  be  measured  in  terms  of  outputs  more  or 
less  immediately  connected  to  the  work.  For  example,  for  some  pur¬ 
poses,  the  value  of  the  development  work  might  be  measured  in  terms 
of  the  technological  parameters  it  affects.  In  that  case,  outputs 
immediately  stemming  from  the  work  measure  its  value.  For  other 
purposes,  the  value  of  the  development  work  might  be  measured  in 
terms  of  its  contribution  to  the  fulfillment  of  a  particular  mission 
or  operational  requirement.  In  those  cases,  results  fairly  far  re¬ 
moved  from  the  actual  work  measure  its  value. 

For  any  single  technological  effort,  therefore,  there  is  gener¬ 
ally  a  function  that  relates  the  relevant  measure  of  expected  results 
to  the  cost  expended  on  the  effort.  A  typical  function  is  shown  in 
Fig.  9;  it  was  developed  during  discussions  at  the  Air  Force  Rocket 
Propulsion  Laboratory  ( AFRPL ) .  It  simply  indicates  that  some  invest¬ 
ment  is  necessary  before  results  of  any  significance  are  obtained; 
this  is  the  TTbuy-inTT  point  on  Fig.  9.  Once  funding  reaches  this  point, 
an  increase  in  value  will  generally  result  from  an  increase  in  effort. 

It  is  difficult  to  measure  with  reasonable  confidence  the  value 
of  most  technical  developments  to  their  end  products  in  operational 
defense  systems. 

A  few  examples  of  such  analyses  have  been  found.  In  HINDSIGHT 
(Ref.  4),  estimates  were  made  of  the  benefits  realized  from  the  intro¬ 
duction  of  two  operational  systems:  the  C-141  cargo  aircraft  and  the 
AN/SPS-48  radar.  A  good  example  of  a  study  of  the  value  of  non- 
Defense  applied  research  is  contained  in  Ref.  18,  an  analysis  of  gold 
recovery  processes  by  the  Department  of  Interior. 
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FIGURE  9.  Typical  Value-Cost  Relationship 


b .  Value  Measures  in  the  Reviewed  Quantitative  Methods .  Out¬ 
side  of  the  Industrial  Analog  and  DOL  methods  which  employ  no  concept 
of  a  value  measure,  three  types  of  value  measures  are  used  in  the 
quantitative  methods  reviewed.  The  first  of  these,  the  type  employed 
in  the  Army  Missile  Plan  and  the  Army  Research  Plan,  is  an  ordinal 
ranking  that  depicts  the  priority  of  each  development  task  according 
to  its  contribution  to  the  weapon  concepts  devised  to  satisfy  the 
various  operational  requirements.  The  second  type  of  value  measure, 
employed  in  TORQUE  and  similar  methods,  is  a  cardinal  measure  that 
attempts  to  quantify  the  contribution  that  completion  of  the  develop¬ 
ment  task  would  make  to  overall  military  utility.  The  third  type  of 
measure  is  the  kind  employed  in  the  Hercules  method,  a  fairly  objec¬ 
tively  estimated  quantity  measuring  the  profits  that  the  company  ex¬ 
pects  to  earn  from  the  commercial  exploitation  of  the  results  of  the 
development  task. 
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On  its  own  terms  and  within  the  limited  objective  of  the  method, 
the  priority  scheme  used  in  the  Army  Missile  Plan  fulfills  its  purpose 
fairly  well.  Priorities  are  assigned  in  the  first  instance  to  the 
individual  weapons  concepts  according  to  a  number  of  criteria,  and 
that  linkage  remains  quite  explicit  throughout  the  remainder  of  the 
planning  process.  Moreover,  these  priorities  are  not  subject  to  sub¬ 
sequent  arithmetic  calculations  that  treat  parts  of  one  ranking  as  if 
they  are  equivalent  to  portions  of  another  ranking.  Consequently,  the 
value  measure  of  the  Missile  Plan  is  quite  visible  to  critics.  Al¬ 
though  it  gives  the  appearance  of  being  rigid  in  application,  it  is 
consistent,  and  it  does  not  give  rise  to  some  of  the  serious  problems 
that  afflict  supposed  cardinal  measures  used  in  other  methods. 

The  second  type  of  value  measure,  which  is  employed  in  TORQUE  and 
the  NOL,  FDL,  and  CAL  methods,  is  an  attempt  to  devise  a  cardinal 
measure  of  the  contribution  that  a  development  effort  would  make  to  an 
indicator  of  overall  military  utility.  In  all  of  these  cases,  resort 
is  made  to  a  consensus  of  subjective  evaluations  of  the  value  question 
by  a  number  of  analysts  and  policy  makers.  The  basic  rationale  for 
collecting  these  subjective  evaluations  and  combining  them  into  the 
value  measures  that  are  finally  used  is  the  Churchman- Ackoff  approxi¬ 
mate  measure  of  value  (Ref.  12,  p.  87). 

Most  of  the  problems  that  arise  in  the  construction  and  applica¬ 
tion  of  these  value  measures  stem  from  the  particular  circumstances 
surrounding  the  assignment  of  value  numbers  to  other  factors  taken 
into  account  in  the  method.  These  are  discussed  in  somewhat  more  de¬ 
tail  under  the  headings  below  relating  to  those  factors. 

In  general,  these  value  measures  suffer  the  same  faults  as  the 
broad  objectives  whose  achievement  they  try  to  gauge.  First,  because 
of  their  highly  subjective  nature,  such  value  measures  should  prove 
to  be  virtually  impossible  to  replicate.  Second,  insofar  as  they 
have  no  counterpart  outside  the  analysts  and  policy  makers  who  assign 
the  numbers  to  the  various  factors  the  value  measures  cannot  be  veri¬ 
fied  in  any  real  sense.  In  other  words,  there  is  just  no  way  to  tell 
whether  the  successful  completion  of  a  single  development  task  or 
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some  set  of  development  tasks  actually  contributes  to  "military 
utility"  to  the  extent  indicated  by  the  assigned  value  measure. 

The  CAL  method  displays  kinship  with  the  TORQUE,  NOL,  and  FDL 
methods  on  this  point  only  in  the  proposal  of  a  similar  value  or 
essentiality  measure.  The  precise  meanings  of  its  "value"  and 
"essentiality"  are  not  made  completely  clear;  much  to  its  credit, 
the  CAL  effort  points  out  the  vagueness  of  these  concepts  and  leaves 
their  measurement  to  further  study. 

The  third  type  of  value  measure  is  exemplified  by  the  Hercules 
method  which  uses  estimates  of  the  profits  that  the  company  will  earn 
from  the  successful  completion  of  a  development  effort  and  commercial 
exploitation  of  its  results .  Replication  of  this  measure  would  appear 
to  be  fairly  easily  accomplished,  and  verification  seems  to  be  no  more 
than  a  matter  of  some  careful  and  purposeful  accounting.  Besides 
having  these  desirable  properties,  company  profits  also  probably 
suffice  as  a  neat  transformation  of  several,  possibly  complex,  value 
measures  into  a  single  indicator  of  the  extent  to  which  the  company’s 
objectives  are  achieved  by  the  development  effort. 

c.  Conclusions .  Obviously,  the  choice  of  the  value  measure  used 
in  the  Hercules  method  depends  upon  the  prior  choice  that  should  have 
been  made  about  the  primary  objective  of  both  the  Exploratory  Develop¬ 
ment  program  and  the  quantitative  method  devised  to  select  the  program. 
However,  while  the  primary  objective  that  is  sought  may  be  acceptable 
for  general  purposes  of  Defense  planning,  the  value  measure  used  to 
indicate  the  extent  of  the  achievement  of  the  objective  may  be  quite 
inappropriate. 

A  value  measure  such  as  that  used  in  the  Hercules  method  just 
does  not  have  a  counterpart  in  Defense  planning  concepts,  or  at  least 
those  Defense  concepts  that  are  similar  are  not  measurable.  In  this 
instance,  Defense  planning,  as  a  whole,  can  be  likened  to  a  barter 
economy  in  which  there  is  no  mechanism  for  transforming  the  units  of 
the  numerous  and  highly  diverse  outputs  into  a  single  measure.  With¬ 
out  a  great  deal  of  progress  in  this  area,  the  value  measure  used  in 
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the  Hercules  method  provides  little  guidance  for  devising  the  compa¬ 
rable  factor  in  a  method  that  could  be  applied  to  allocating  the 
budget  for  Defense  Exploratory  Development. 

Although  more  specific  problems  of  the  value  measures  employed 
in  the  TORQUE,  NOL,  and  FDD  methods  are  discussed  below,  a  general 
objection  to  them  is  that  they  are  extremely  nebulous.  They  are 
necessarily  so  because  of  what  they  are  purported  to  measure,  vague 
expressions  of  overall  military  utility.  The  impossibility  of  verify¬ 
ing  these  measures  and  their  particular  assignment  to  specific  develop¬ 
ment  efforts  makes  their  usefulness  highly  dubious,  at  best.  More¬ 
over,  to  anticipate  some  objections  raised  below,  the  application  of 
the  particular  procedure  adopted  to  synthesize  these  value  measures 
does  not  make  them  proxies  for  a  value  measure  paralleling  the  profit 
concept  employed  in  the  Hercules  method. 

Inasmuch  as  the  inappropriateness  of  many  of  the  value  measures 
stems  from  the  overly  ambitious  objectives  set  for  the  quantitative 
methods  in  which  they  are  used,  analysts  should  try  to  specify  more 
modest  objectives  and  value  measures,  at  least  at  this  stage  of 
progress.  Somewhat  more  modest  objectives  have  been  discussed  above. 
Measuring  the  cost  of  a  development  effort  or  its  technical  comple¬ 
tion  should  prove  to  be  much  more  easily  verified  than  any  utility 
scale  that  has  been  used  as  a  value  measure.  Moreover,  the  simplicity 
and  directness  of  a  priority  scaling,  such  as  that  used  in  the  Army 
Missile  Plan,  are  highly  desirable,  especially  if  the  scaling  is  not 
subjected  to  a  number  of  subsequent  arithmetic  manipulations  that 
obscure  its  limited  meaning. 

More  intensive  research  should  be  undertaken  on  this  factor. 

Any  progress  in  devising  appropriate  measures  of  value  for  Defense 
planning  would  be  helpful.  The  approach  to  measuring  cost  savings 
adopted  in  HINDSIGHT  might  be  worthwhile  to  extend  to  situations  in 
which  the  basic  mission  is  changed.  Approaches  that  would  display 
the  trade-offs  among  a  number  of  measures  of  value  should  be  examined 
as  well. 


2.  Operational  Requirements 


a.  Discussion .  Operational  requirements  emanate  from  a  joint 
consideration  of  the  range  of  threats  that  might  be  made  to  the  mili¬ 
tary  security  of  the  United  States  and  the  military  capabilities  that 
are  needed  to  counter  those  threats.  A  uniform  set  of  operational  re¬ 
quirements  could  be  defined  for  DoD-wide  planning;  probably  less  than 
100  would  suffice.  Each  operational  requirement  should  give  sufficient 
information  for  the  systems  designers  who  use  them  to  understand  the 
performance  characteristics  they  must  try  to  incorporate  into  any 
equipment . 

Senior  level  personnel  from  the  Services  and  the  OSD  should 
undertake  the  definition  of  these  requirements  so  that  they  carry 
sufficient  authority  and  uniformity  to  encourage  their  adoption. 
Definition  of  the  requirements  by  such  personnel  might  also  prevent 
the  development  of  an  undesirable  informal  division  of  them  among  the 
Services  where  benefits  of  strict  specialization  are  not  particularly 
clear. 

If  there  is  unanimity  on  the  composition  of  the  threat  to  the 
military  security  of  the  U.S.,  the  operational  requirements  should 
reflect  the  balanced  goals  that  are  to  be  satisfied  simultaneously  in 
some  particular  future  period.  Possibly  more  than  one  set  of  require¬ 
ments  could  satisfy  both  these  military  goals  and  other  national  goals 
at  the  same  time  so  that  different  sets  might  be  specified  depending 
upon  the  time  period  or  other  conditions  considered. 

Operational  requirements  can  probably  be  developed  from  existing 
Service  documentation  of  desired  military  capabilities  such  as  OCOs, 
GORs,  desired  capabilities,  etc.  Currently  the  development  of  Service 
military  capabilities  often  takes  a  long  time.  Figure  10  shows  the 
steps  in  the  development  of  an  Army  QMDO. 

In  any  event,  specification  of  the  operational  requirements 
should  take  place  outside  the  planning  of  the  Exploratory  Development 
program.  Proper  definition  of  these  requirements  is  one  of  the  ways 
by  which  common  ground  rules  can  be  imposed  on  the  various  stages  of 
the  overall  development-procurement-operations  process. 
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FIGURE  10.  Requirements  Procedure,  Army  QMDO 

b .  Operational  Requirements  in  the  Reviewed  Quantitative  Methods . 
All  of  the  methods  that  were  reviewed  use  some  factor  that  parallels 
the  concept  of  an  operational  requirement.  For  present  purposes,  the 
military  requirements  and  the  weights  assigned  them  in  the  derivation 
of  the  value  measure  are  of  primary  interest. 

In  an  experimental  application  of  TORQUE,  a  set  of  13  operational 
requirements  were  developed  for  the  Air  Force  (Fig.  11).  The  basic 
formulation  of  the  TORQUE  method  calls  for  precise  definitions  of  the 
missions  that  the  Defense  Department  is  expected  to  perform  in  support 
of  national  security  goals  and  for  a  systematic  clarification  of  the 
relationships  among  those  missions.  However,  a  problem  can  arise  re¬ 
garding  analytical  applications  of  the  definitions  if  an  attempt  is 
made  to  assign  measures  of  relative  importance  (value)  to  the  opera¬ 
tional  requirements .  When  analysis  calls  for  assignments  of  quanti¬ 
tative  measures  of  value  to  the  requirements,  the  properties  of  the 
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DESTROY  ENEMY  STRATEGIC  FORCES  AND  RESOURCES 
DESTROY  ATTACKING  AIRBREATHING  FORCES 
DESTROY  ATTACKING  BALLISTIC  MISSILES 
DESTROY  HOSTILE  SPACE  FORCES 

CLOSE  AIR  SUPPORT 

DEEP  STRIKE/INTERDICTION 

COUNTERAIR 

SPECIAL  AIR  OPERATIONS 

SURVEILLANCE/RECONNAISSANCE 

COMMAND/CONTROL/COMMUNICATIONS 

AIRLIFT 

NAVIGATION  AIDS 
SPACE  TRANSPORTATION 


FIGURE  11  .  Desired  Capabilities,  Air  Force 


analytical  method,  the  value  measures,  and  the  real  counterparts  of 
the  definitions  must  be  consistent.  Analogously,  the  speed  of  an 
automobile  cannot  be  gauged  by  a  lineal  measuring  device  alone;  a 
timer  is  also  required,  and  these  measures  must  be  integrated  prop¬ 
erly  if  they  are  to  be  used  in  an  accurate  display  device. 

Consequently,  the  originators  of  TORQUE  recognized  that  the  re¬ 
quirements  must  possess  three  characteristics;  (1)  they  should  be 
relatively  stable  in  definition  and  importance  over  time,  (2)  they 
should  be  defined  independently  of  actual  and  prospective  technology 
developments,  and  (3)  they  should  be  independent  of  each  other.  The 
first  two  of  these  properties  are  necessary  for  the  requirements  to 
fit  into  any  analytical  or  allocation  framework.  The  first  ensures 
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•  GENERAL  WAR 

•  TACTICAL  WAR 
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•  GENERAL  SUPPORT 


that  there  is  sufficient  continuity  in  the  problem  structure  that 
analysis  can  be  performed,  verified,  implemented,  and  eventually  tested 
for  the  correctness  of  the  results.  The  second  ensures  that  the  goals 
themselves  are  not  dependent  upon  the  manner  in  which  the  development 
resources  are  expended.  Such  dependence  would  only  lead  to  a  circular 
planning  process  that  would  eventually  become  self- confirming.  The 
third  characteristic  stems  directly  from  the  value  measure  that  the 
originators  recommend  be  applied  to  the  requirements . 

In  the  TORQUE  procedure,  one,  and  only  one,  relative  weight  is 
assigned  to  a  particular  operational  requirement.  For  example,  the 
one  specifying  the  destruction  of  enemy  strategic  forces  and  resources 
might  be  assigned  the  highest  weight,  say  100,  among  the  Air  Force  re¬ 
quirements.  At  the  minimum,  this  unique  assignment  confounds  two  sep¬ 
arate  considerations:  the  importance  of  the  general  mission  and  the 
probable  performance  of  the  mission.  Several  gradations  appear  to  be 
possible  in  the  performance  of  this  type  of  mission,  depending  upon 
the  location  of  the  enemy,  the  types  of  forces  and  resources  that  are 
to  be  destroyed,  and  the  amounts  of  these  forces  and  resources  that 
are  likely  to  be  destroyed.  The  single  measure  of  value  for  the 
mission  results  in  different  capabilities  in  these  missions  being 
treated  as  though  they  were  equally  important. 

The  TORQUE  manuals  recommend  the  application  of  the  Churchman- 
Ackoff  approximate  measure  of  value  to  the  operational  requirements . 

It  appears  to  generate  the  types  of  weights  that  could  be  used  in  the 
TORQUE  ’’utility  function.” 

The  Churchman- Ackoff  measure  (cf.  Ref.  12,  pp.  87-91)  is  based 
on  a  special  procedure  for  assigning  value  quantities  to  the  occur¬ 
rence  of  various  events .  Basically,  it  compares  the  events  pairwise 
and  in  various  (and  any)  combinations  to  solve  for  a  consistent  (but 
not  necessarily  unique)  weight  for  each  event  that  will  preserve  the 
rankings  of  the  events  in  any  combination .  To  fit  this  kind  in 
measurement  framework,  the  actual  events  must  be  of  a  special  type. 
Logically,  they  must  be  able  to  occur  simultaneously,  and  they  must 
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be  independent  of  each  other.  If  two  particular  events  could  not 
occur  at  the  same  time,  they  and  their  weights  could  not  be  combined 
for  comparison  with  that  of  other  events;  therefore,  the  procedure  for 
determining  consistent  weights  could  not  be  carried  out.  If  two  par¬ 
ticular  events  were  not  independent  of  each  other,  the  relative  impor¬ 
tance  of  their  simultaneous  occurrence  would  not  be  the  same  as  the 
sum  of  their  individual  weights  when  they  occur  separately.  Conse¬ 
quently,  the  assignment  of  individual  weights  through  pairwise  and 
combination  event  comparisons  could  not  be  done. 

Although,  in  general,  the  operational  requirements  can  be  defined 
in  any  way  that  meets  the  needs  of  management  or  analysis,  the  proper¬ 
ties  required  of  the  subject  events  for  application  of  the  Churchman- 
Ackoff  measure  must  be  met,  or  the  procedure  for  assigning  weights  be¬ 
comes  virtually  meaningless. 

The  consequences  of  these  properties  for  the  operational  require¬ 
ments  and  the  TORQUE  method  in  general  are  quite  significant.  First, 
because  of  their  very  broad  definitions  and  specific  contents,  each 
of  the  requirements,  as  they  currently  stand,  can  hardly  be  treated  as 
individual  events.  The  achievements  possible  under  any  one,  such  as 
the  Air  Force  desired  capabilities  listed  in  Fig.  11,  are  so  diverse 
and  variable  that  any  specific  combinations  of  such  achievements  can¬ 
not  be  taken  to  be  alternative  events  of  equal  value.  Would  the 
destruction  of  a  factory  that  manufactures  ICBMs  be  the  equivalent  of 
destroying  ICBMs  deployed  and  ready  in  their  silos?  Both  of  these 
cases  appear  to  fall  under  the  one  operational  requirement,  "Destroy 
Enemy  Strategic  Forces  and  Resources." 

At  the  same  time,  however,  breaking  out  separable  events  from  the 
general  requirement  may  only  shift  the  source  of  difficulty  with  the 
measure  for  these  purposes.  The  events  so  devised  by  definition  must 
still  correspond  to  real  counterparts  that  are  able  to  occur  simul¬ 
taneously,  and  these  real  counterparts  must  be  practically  independent. 
Such  conditions  can  be  extremely  difficult  to  satisfy.  And  the  inde¬ 
pendence  requirement  is  particularly  severe. 
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Independence  of  the  requirements  means  that  the  fulfillment  of 
any  two  of  them  jointly  generates  the  same  value  as  the  sum  of  the 
values  that  would  result  from  their  separate  fulfillment.  This  con¬ 
dition  can  be  illustrated  in  terms  of  the  requirements  listed  in 
Fig.  11.  Taking  two  of  the  requirements,  close  air  support  and 
command /control/communications ,  three  separate  ’’events”  are  possible: 
(1)  the  Air  Force  supplies  close  air  support  with  random  sorties  and 
strictly  by  pilot  judgment,  (2)  the  Air  Force  maintains  a  command/ 
control/communications  network,  and  (3)  the  Air  Force  supplies  close 
air  support  combined  with  a  command /control/communications  network. 

If  the  requirements  were  independent,  the  value  of  Event  3  would  be 
just  equal  to  the  separate  value  of  Event  1  plus  the  separate  value 
of  Event  2.  While  the  isolated  use  of  close  air  support  may  supply 
some  defense  effectiveness  and  the  isolated  use  of  a  command /control/ 
communications  network  may  generate  some  defense  benefits,  the  joint 
application  of  these  functions  would  appear  to  have  a  much  greater 
effectiveness  than  a  simple  sum  of  their  isolated  benefits  because 
they  complement  each  other. 

This  problem  is  then  carried  through  the  remainder  of  the  steps 
in  the  method  that  depend  upon  the  weights  assigned  the  operational 
requirements . 

Although  the  terminology  employed  is  somewhat  different,  the  NOL 
and  FDL  methods  rely  upon  conceptual  constructs  that  are  much  like 
the  operational  requirements  concepts  used  in  TORQUE.  These  methods 
also  propose  the  use  of  the  Churchman- Ackoff  approximate  measure  of 
value  as  the  procedure  that  should  be  followed  to  assign  importance 
weights  to  their  requirements  concepts.  Consequently,  their  weighting 
schemes  suffer  the  same  deficiencies  as  those  described  above  for 
TORQUE . 

c.  Conclusions .  Obviously,  the  definition  of  operational  re¬ 
quirements  can  be  an  extremely  helpful  step  in  the  planning  process. 

If  carried  out  clearly  and  with  precision,  these  definitions  serve  to 
break  out  higher  order  objectives  into  components  that  are  more 


103 


specific  and  operational.  However,  a  number  of  problems  can  arise  in 
the  attempt  to  make  operational  requirements  explicit.  One  of  these 
problems  is  finding  the  proper  level  of  personnel  with  expertise  and 
perspective  to  set  out  the  operational  requirements  so  that  the 
Services  and  OSD  can  agree  on  the  content  of  a  common  set  to  be  used 
by  all  elements  of  the  Department  of  Defense.  Another  problem  is  the 
specification  of  an  exhaustive  set  of  requirements  that  will  adequately 
cover  all  of  the  functions  carried  out  in  Defense.  For  example,  the 
operational  requirements  set  out  for  the  experimental  application  of 
TORQUE  by  the  Air  Force  and  the  QMDOs  used  by  the  Army  do  not  appear 
to  include  items  that  express  nonhardware  types  of  objectives  such 
as  might  be  dealt  with  by  behavioral  sciences. 

Another  serious  problem  revealed  in  the  reviews  concerns  the 
assignment  of  ’’weights”  to  the  various  operational  requirements  to 
form  measures  of  ’’military  utility.”  In  some  of  the  methods,  the 
weighting  was  necessary  to  the  operation  of  the  allocation  technique. 
However,  the  significance  of  the  weights  of  operational  requirements 
and  the  procedures  for  assigning  them  do  not  appear  to  be  uniformly 
understood.  For  example,  different  weights  might  be  taken  to  mean 
(1)  the  manager  is  willing  to  incur  higher  costs  to  fulfill  one  re¬ 
quirement  compared  to  another,  (2)  the  manager  considers  one  require¬ 
ment  just  absolutely  more  important  than  another  by  some  scale  measure, 
or  (3)  the  manager  would  like  to  fulfill  the  defined  operational  re¬ 
quirement,  if  possible,  before  fulfilling  others  of  lesser  rank. 

These  meanings  obviously  confound  a  number  of  considerations 
dealing  with  both  the  importance  of  the  requirements  and  the  costs  of 
fulfilling  them.  But  even  if  the  criteria  of  importance  could  be 
made  precise,  the  problem  remains  that  the  method  used  to  assign  the 
weights  pertains  to  a  very  special  type  of  event  whose  features  are 
not  very  closely  matched  by  operational  requirements. 

3 .  Weapon  Systems 

a.  Discussion.  Devising  weapon  system  concepts  is  another  step 
in  the  procedure  for  planning  Exploratory  Development  that  moves  from 
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the  more  general  military  objectives  to  the  more  specific  content  of 
the  technological  advances  needed.  In  this  step,  analysts  try  to 
design,  more  or  less  precisely,  the  hardware  that  could  fulfill  the 
operational  requirements  in  some  way. 

Appraisal  of  the  future  impact  of  a  technological  advance  depends 
heavily  upon  projections  of  the  hypothetical  systems  that  will  utilize 
the  technology.  During  the  Field  Survey  some  Service  laboratory 
organizations  were  found  to  ignore  clearly  identified  probable  future 
systems;  this  did  not  seem  to  prevent  high  quality  efforts  in  Explora¬ 
tory  Development.  However,  the  existence  of  probable  future  systems 
in  mind  if  not  on  paper  usually  became  apparent  in  the  course  of 
discussions . 

The  responsibility  for  the  development  of  the  probable  future 
systems  is  usually  assigned  to  the  System  Commands  in  the  Services; 
the  laboratories,  that  ultimately  manage  the  Exploratory  Development 
effort,  participate  in  the  synthesis  of  probable  future  systems  at  a 
relatively  low  level.  The  Army  Missile  Command  performs  system 
studies  both  in-  and  out-of-house :  recent  studies  include  SAM  D 
and  the  LAW  (light  antitank  weapon)  Workshop;  future  studies  include 
TRAADS  and  MARS  (multiple  artillery  rocket  system). 

Many  examples  of  present  ability  to  translate  operational  re¬ 
quirements  into  probable  future  systems  have  been  found  in  all  three 
Services.  The  Navy  has  probably  advanced  the  art  of  presenting  all 
probable  future  systems  more  than  any  other  Service.  The  Navy  Tech¬ 
nological  Forecast  (Ref.  19)  presents  probable  future  systems  and 
subsystems  for  the  entire  Navy.  Earlier  efforts  within  the  Navy  were 
centered  in  the  Naval  Air  Systems  Command  with  the  publication  of  the 
Annual  SMEADO  (Selected  Major  Exploratory  and  Advanced  Development 
Objectives)  and  the  Naval  Ordnance  Systems  Command  ACORD.  The  field 
survey  also  found  that  three  or  four  systems  may  typically  describe 
most  of  the  reasonable  approaches  for  a  single  operational  require¬ 
ment  and  can  be  substantiated  by  good  to  fair  systems  analysis. 


In  the  view  of  some  of  the  Exploratory  Development  managers  who 
have  had  probable  future  systems  made  available  to  them,  there  is  an 
impression  that  the  system  characteristics  change  rapidly  in  compari¬ 
son  to  the  ability  to  reorient  Exploratory  Development  programs. 

During  the  field  survey,  it  was  also  found  that  there  was  great 
variation  in  the  quality  of  the  systems  analysis  supporting  the 
probable  future  systems . 

A  good  systems  analysis  to  meet  an  increased  threat  will  compare 
the  expansion  of  present  systems,  as  well  as  consider  alternatives  in 
new  development  to  satisfy  needs.  It  will  also  consider  operations 
and  maintenance  in  arriving  at  the  recommended  probable  future  systems. 
Even  in  the  relatively  mature  Army  Missile  Plan  it  is  estimated  that 
only  about  one-third  of  the  candidate  systems  are  substantiated  by  a 
high-quality  systems  analysis. 

One  way  to  deal  with  the  interface  between  Exploratory  Develop¬ 
ment  and  the  overall  sequence  of  development,  procurement,  and  opera¬ 
tions  is  through  functional  relationships  among  factors  such  as  those 
pertaining  to  weapon  systems. 

In  a  quantitative  method  supporting  systems  development,  func¬ 
tional  relationships  can  be  used  to  make  explicit  the  assumptions 
that  must  be  imposed  on  Exploratory  Development  from  the  larger  con¬ 
text  about  (1)  the  performance  characteristics  of  alternative  weapon 
systems  proposed  to  serve  the  same  operations  requirement  and  (2)  the 
conditions  that  must  hold  among  elements  of  different  force  struc¬ 
tures  and  different  force  levels  to  satisfy  the  higher  order  military 
objectives . 

It  is  possible  that  quantitative  methods  for  allocating  resources 
in  Exploratory  Development  are  suited  to  the  more  system-oriented 
Services  like  the  Navy  and  Air  Force  but  not  to  a  doctrine -dominated 
Service  like  the  Army.  Such  a  conclusion  might  be  based  upon  the 
efforts  by  the  Army  in  1967  to  develop  operational  requirements  that 
could  lead  to  systems .  This  effort  paralleled  the  one  in  the  Air  Force 
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that  ultimately  led  to  the  TORQUE  experiment  at  AFFDL.  It  should 
be  noted,  however,  that  in  our  very  limited  field  survey  no  such 
difficulty  was  reported,  either  in  the  fields  of  rocket  propulsion 
or  Army  electronic  components  research., 

bo  Weapon  Systems  in  the  Reviewed  Quantitative  Methods 9  All 
of  the  Quantitative  methods  reviewed  rely  upon  weapon  system  designs, 
to  some  extent,  as  intermediate  planning  concepts  to  determine  the 
technological  advances  that  Exploratory  Development  efforts  should 
try  to  attain o  However,  the  ways  in  which  the  various  methods  use 
the  designs  differ  fairly  markedly 9 

Some  of  the  methods,  such  as  the  Industrial  Analog  and  the  NOL 
methods,  incorporate  considerations  of  systems  designs  only  implicitly. 
Others  require  a  very  explicit  enumeration  of  the  prospective  weapon 
systems  along  with  fairly  specific  designs. 

The  Industrial  Analog  method  proposed  to  consider  product  com¬ 
plexities  and  lifetime  in  its  synthesis  of  factors  for  determining  the 
distribution  of  the  Exploratory  Development  effort.  However,  much  of 
the  information  on  commercial  and  Defense  product  complexity  and  life¬ 
times  is  necessarily  a  transformation  of  ordinal  human  judgments  to 
cardinal  scaling.  Little  has  been  reported  in  the  Industrial  Analog 
material  on  studies  of  the  complexity  of  the  products  in  the  technical 
fields.  Moreover,  regarding  product  lifetimes,  major  questions  arise 
over  product  definition  and  whether  modifications  merely  extend  the 
lives  of  some  products  or  whether  the  modifications  result  in  new 
products . 

The  NOL  method  relies  upon  only  a  general  functional  analysis  of 
weapon  systems  that  could  fulfill  the  operational  requirements .  These 
are  incorporated  into  the  Exploratory  Development  goals . 

In  TORQUE,  members  of  the  Interdisciplinary  Team  design  weapon 
systems  to  fulfill  the  various  operational  requirements «  In  devis¬ 
ing  the  weapon  systems,  the  systems  analysts  are  not  compelled  by 
the  TORQUE  procedure  to  compare  the  existing  force  structure,  possible 
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changes  in  tactics,  possible  modifications  to  the  existing  force 
structure  and  level,  and  potential  new  forces.  Moreover,  the  TORQUE 
analytical  framework  does  not  evaluate  these  options  in  terms  of 
operational  requirement  performance  and  full  costs.  Consequently, 
it  leaves  the  uneasy  feeling  that  these  factors  could  well  be  ignored 
or  that' they  have  not  been  given  reproducible  and  verifiable 
scrutiny. 

Just  as  variations  in  the  performance  of  an  operational  require¬ 
ment  are  not  recognized  as  separate  events  in  the  assignment  of  utility 
values,  the  different  performance  levels  of  the  various  systems  em¬ 
ployed  in  a  requirement  are  not  recognized.  Each  system  applied  to  a 
given  operational  requirement  is  given  full  credit  for  the  require¬ 
ment's  fulfillment.  Undoubtedly,  all  systems  that  can  be  devised  for 
a  particular  requirement  will  not  perform  in  the  same  manner.  By 
forcing  all  systems  into  this  performance  structure,  TORQUE  may  be 
denying  the  technology  and  weapons  managers  the  very  real  flexibility 
of  being  able  to  trade  off  some  performance  (and  possibly  some  time 
to  IOC)  for  costs . 

Moreover,  once  the  weapon  systems  are  devised  and  introduced  into 
the  TORQUE  framework,  they  are  not  treated  as  alternative  means  of 
supporting  particular  operational  requirements .  It  is  as  though  each 
requirement  is  insatiable,  the  deployment  of  any  additional  weapon 
system  in  a  particular  requirement's  arsenal  adding  as  much  to  its 
achievement  as  if  the  weapon  were  the  first  and  only  one  in  that 
arsenal. 

The  formulators  of  the  FDL  method  have  tried  to  get  around  some 
of  these  objections  to  the  treatment  of  weapon  systems  in  TORQUE. 
Analysts  assign  values  to  the  various  weapon  systems  according  to  the 
contributions  they  make  to  the  different  operational  requirements. 
However,  the  analysts  continue  to  treat  the  weapon  systems  as  though 
they  are  perfectly  independent  of  each  other  in  contributing  to  opera¬ 
tional  requirements.  Such  independence  is  highly  unlikely.  There  is 
no  demonstration  that  various  combinations  of  weapon  systems  should 
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have  weights  equal  to  the  sum  of  the  separate  weapons  in  those  com¬ 
binations.  Moreover,  there  is  no  inherent  limit  upon  the  number  of 
weapon  systems  that  might  be  applied  to  any  single  operational 
requirement . 

In  the  strict  construction  of  the  CAL  method,  the  military 
’’essentiality”  of  the  QMDO  is  the  basis  for  the  measure  of  value  used 
in  the  objective.  This  measure  is  replete  with  problems  similar  to 
those  that  affect  value  measures  of  the  TORQUE  and  NOL  methods. 

Although  the  method  formulation  might  be  appropriately  modified  and 
satisfactory  value  measures  estimated  for  the  QMDOs ,  the  form  re¬ 
viewed  treats  the  QMDOs  as  though  they  are  independent  of  one  another. 

Further,  each  materiel  concept  in  the  CAL  method  is  a  necessary 
condition  of  the  fulfillment  of  the  QMDO  for  which  it  is  a  component. 

The  materiel  concept  itself  consists  of  a  set  of  critical  performance 
parameters .  Such  a  formulation  appears  to  be  undesirably  rigid . 
Trade-offs  among  performance  parameters  to  maintain  the  intended 
effectiveness  of  the  final  weapon  system  configuration  should  be  pos¬ 
sible  within  any  one  materiel  concept.  Similarly,  trade-offs  among 
the  performance  characteristics  of  the  different  subsystems  should  be 
possible  while  preserving  the  final  system  effectiveness .  Explicit 
formulation  of  these  dimensions  of  variation  would  add  significantly 
to  the  options  available  to  management  and  probably  lead  to  a  better 
assessment  of  the  overall  probabilities  of  success  of  a  program.  Such 
a  formulation  should  also  provide  avenues  for  planning  lower  cost 
programs  that  would  achieve  given  military  objectives. 

Systems  (and  other  types  of  follow-on  contracts)  play  a  pivotal 
role  in  the  Hercules  method.  The  marketing  personnel  calculate  poten¬ 
tial  sales  and  profits  that  Exploratory  Development  might  generate  on 
the  basis  of  the  systems  incorporating  the  projected  technological 
advance . 

In  the  Army  Missile  Plan,  weapon  systems  are  enumerated  explicitly 
as  the  means  for  fulfilling  operational  requirements.  The  identifica¬ 
tion  of  the  weapon  systems  is  preserved  and  the  connection  of  any  single 
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weapon  system  to  a  specific  operational  requirement  remains  visible 
throughout  the  subsequent  planning  steps . 

c.  Conclusions .  Except  in  a  commercial  application,  such  as  the 
Hercules  method,  or  in  an  application  of  modest  scope,  such  as  the 
Army  Missile  Plan,  weapon  systems  concepts  do  not  appear  to  be  treated 
adequately  in  the  reviewed  quantitative  methods .  Measures  of  the  value 
of  weapon  systems  (with  regard  to  the  achievement  of  operational  ob¬ 
jectives)  ignore  (1)  the  possibility  that  operational  requirements  may 
be  fulfilled  in  different  ways  and  to  varying  extents,  (2)  the  possible 
interactions  among  systems  when  used  in  combination,  (3)  the  substi¬ 
tutability  of  proposed  weapon  systems  for  fulfilling  an  operational 
requirement,  and  (4)  the  performance  of  proposed  systems  relative  to 
possible  variations  in  current  force  mixes  and  levels. 

4.  Technologies 

a.  Discussion.  In  quantitative  methods  that  are  used  to  plan 
Exploratory  Development  programs  supporting  weapon  system  development, 
the  factor  generally  considered,  once  prospective  systems  are  speci¬ 
fied,  is  the  technological  composition  of  the  systems.  The  techno¬ 
logical  composition  breaks  down  the  system  design  into  the  existing 
or  projected  engineering  characteristics  that  the  weapon  system  must 
contain  if  it  is  to  perform  in  the  way  desired. 

Functional  relationships  involving  this  factor  trace  the  substi¬ 
tutions  that  might  be  made  among  technologies  and  possibly  among  some 
component  operating  characteristics  across  a  number  of  system  designs 
having  the  same  overall  performance  qualities.  These  relationships 
also  demonstrate  how  technologies  may  complement  one  another;  that  is, 
how  some  technologies  must  be  used  in  combination  in  particular  weapons 
systems  to  fulfill  particular  performance  qualities. 

Such  relationships  should  facilitate  tracing  changes  in  the  per¬ 
formance  of  a  weapon  system  that  take  place  as  changes  are  made  in  the 
parameters  of  the  component  technologies. 
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Two  levels  of  analysis  must  be  performed  within  individual  systems 
to  determine  the  contributions  of  the  subsystems  and  the  component 
technologies.  The  first  and  higher  level  is  that  which  must  be  per¬ 
formed  within  the  system  to  compare  major  subsystems,  e.g. ,  to  deter¬ 
mine  the  importance  of  the  guidance  of  a  ballistic  missile  as  opposed 
to  the  propulsion  system.  Such  analyses  would  normally  be  performed 
by  or  under  the  cognizance  of  the  Systems  Command. 

The  second  type  of  analysis  is  performed  within  a  particular  sub¬ 
system  to  identify  the  impact  of  various  technologies.  These  would 
normally  be  prepared  or  sponsored  by  the  Service  laboratories.  During 
the  field  survey,  examples  of  good  work  of  this  type  were  identified, 
but  they  did  not  appear  to  be  widespread.  In  rocket  propulsion  the 
Navy  cited  detailed  studies  such  as  those  performed  by  the  Naval  War¬ 
fare  Center  on  Advanced  Tactical  Stand-off  ASM  Systems  which  permitted 
the  establishment  of  values  for  jinking,  thrust  modulation,  and  ramjet 
angle  of  attack  sensitivity.  The  Army  MICOM  Propulsion  Laboratory  cited 
the  SHORAD  study  in  which  the  sensitivity  factors  of  18  items  were 
determined  (smokeless  characteristics,  diameter,  weight,  hot  atmosphere 
effect,  etc.).  Analyses  to  evaluate  more  qualitative  factors  such  as 
propellant  storage  characteristics  are  few  and  difficult  to  perform. 

Estimates  were  made  by  the  Service  Propulsion  Laboratories  of  the 
proportion  of  manpower  devoted  to  studies  of  the  second  level  mentioned 
above,  i.e.,  analysis  within  subsystems.  For  the  Army,  Navy,  and  Air 
Force  the  percentages  were  5,  2,  and  3,  respectively. 

b.  Technologies  in  the  Reviewed  Quantitative  Methods.  Some  of 
the  methods  reviewed  do  not  derive  their  proposed  technological  effort 
from  fairly  concrete  future  weapon  systems.  The  Industrial  Analog 
method,  for  example,  proposes  to  relate  current  military  Exploratory 
Development  effort  in  a  technology  to  the  company- financed  effort  on 
the  same  technology  in  American  industry.  Also,  the  NOL  method  does 
not  require  fairly  specific  weapon  system  concepts  to  derive  a  desired 
technological  effort. 


Ill 


One  disquieting  feature  of  the  Industrial  Analog  is  that  its 
functional  relationships  describe  historical  conditions,  and  do  not 
directly  measure  or  project  a  connection  between  current  R&D  effort 
and  currently  demanded  technological  developments .  Consequently, 
the  allocation  system  contains  a  bias  favoring  continued  work  in  the 
pattern  of  technical  fields  that  supplied  inputs  to  the  weapon  systems 
in  the  existing  or  recent  past  defense  force  structure.  The  effect 
that  changes  in  the  projected  threat  might  have  on  the  more  basic 
categories  of  the  research  and  development  program  under  the  Industrial 
Analog  planning  procedure  is  not  clear. 

In  the  NOL  method,  the  analysts  resort  to  the  Exploratory  Devel¬ 
opment  goals  to  describe  the  work  that  must  be  done  at  the  Exploratory 
Development  level  to  satisfy  operational  requirements  of  future  weapons 
and  support  systems,  but  these  are  not  specific  designs.  Subsequently, 
each  technology  is  assigned  a  "utility"  index  based  upon  the  relevance 
of  that  technology  to  the  EDG.  However,  the  particular  linkages  of  the 
technologies  to  the  EDGs  appear  to  ignore  the  substitutions  that  might 
be  possible  among  the  technologies  and  the  complementarities  that  exist 
among  them  with  regard  to  the  EDG.  Moreover,  there  does  not  appear 
to  be  any  strict  account  taken  of  the  level  of  the  pacing  parameter 
of  the  technology  that  would  be  needed  to  fulfill  any  single  EDG. 

The  other  methods  derive  the  technologies  to  be  included  in  the 
Exploratory  Development  program  more  directly  from  specific  weapon 
system  designs. 

In  TORQUE,  for  example,  the  profile  of  technology  objectives  that 
makes  up  any  weapon  system  is  expressed  through  the  "criticality" 
value  that  the  I.D.  team  assigns  each  technology  objective  with  regard 
to  each  weapon  system.  These  criticalities  are  largely  subjective 
factors  that  confound  at  least  three  separable  features:  (1)  the 
extent  to  which  subsystems  or  technology  objectives  can  be  substi¬ 
tuted  for  each  other  while  maintaining  equal  effectiveness  in  a  par¬ 
ticular  weapon  system,  (2)  the  cost  implications  of  using  specific 
technology  objectives  in  a  particular  weapon  system,  and  (3)  the 
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variations  in  the  performance  characteristics  of  a  weapon  system  that 
might  result  from  the  use  of  some  technology  objectives. 

These  features  should  not  be  mixed  up  in  this  way;  the  first  is  a 
technological  input  trade-off  measure,  the  second  is  an  economic  meas¬ 
ure,  and  the  third  is  a  measure  of  the  output  quality  trade-offs.  In 
other  words,  the  criticalities  mix  ’’apples  and  oranges’'  without  any 
correspondence  to  the  real  counterparts  of  the  relevant  measures  and 
trade-offs  that  are  involved. 

Moreover,  as  the  criticality  measures  are  used,  they  do  not  modify 
the  effectiveness  or  weight  attributed  to  a  weapon  system  if  some,  but 
not  all,  of  the  component  technology  objectives  are  included  in  the 
Exploratory  Development  program . 

None  of  the  functional  relationships  in  the  TORQUE  analytical 
framework  incorporate  explicitly  any  measure  of  the  extent  to  which 
one  technology  objective  may  be  substituted  for  another.  Therefore, 
the  method  does  not  consider  the  particular  technology  objectives  that 
can  be  substituted  for  one  another  to  be  alternatives  for  completing 
a  weapon  system  or  0C0.  In  fact,  if  more  than  one  technology  objective 
could  fulfill  a  particular  function  but  only  one  must  be  developed, 
the  current  structure  of  TORQUE  would  make  all  appear  to  be  equally 
"critical 

Complementary  technology  objectives  are  those  that  are  related 
to  one  another  and  mutually  supportive  in  a  particular  weapon  system. 
The  current  TORQUE  framework  does  not  include  functional  relationships 
that  incorporate  such  complementarities  into  the  considerations  that 
must  be  taken  into  account. 

Despite  the  intricate  synthesis  of  information  in  the  method, 
each  technology  objective  is  treated  as  an  independent  effort  in 
TORQUE.  Consequently,  the  method  does  not  treat  as  alternatives 
those  technology  objectives  that  are  substitutes  for  one  another.  Nor 
does  it  treat  as  complements  those  technology  objectives  that  mutually 


support  each  other  in  particular  weapon  system  configurations.  In  the 
current  form  of  the  ’’utility  function,”  pursuit  of  a  particular  tech¬ 
nology  objective  does  not  diminish  the  utility  contribution  of  a  sub¬ 
stitute  that  could  be  added  to  the  program  without  deletion  of  the 
first.  Also,  the  utility  function  does  not  recognize  that  once  a 
particular  technology  objective  is  chosen,  its  complements  become 
’’more  important.” 

Within  the  relationships  of  the  method,  therefore,  it  is  quite 
possible  to  choose  a  set  of  redundant  technology  objectives  and,  at 
the  same  time,  a  patchwork  of  technology  objectives  that  would  not 
make  up  a  single  viable  weapon  system. 

These  observations  apply  equally  to  the  applicability  factor 
employed  in  the  FDL  method. 

On  the  other  hand,  the  CAL  method  does,  to  some  extent,  take  into 
account  the  possible  substitutions  and  complementary  relationships 
among  technical  objectives.  This  is  accomplished  through  the  substitu¬ 
tions  that  can  be  made  among  the  alternative  technical  approaches  to  a 
particular  materiel  concept  and  the  strong  complementary  relationships 
among  the  materiel  concepts  of  a  given  QMDO.  However,  the  CAL  method 
does  not  extract  the  common  technical  goals  that  might  be  involved  in 
several  QMDOs.  Each  technical  accomplishment  contained  in  a  QMDO  is 
treated  as  though  it  is  a  component  of  only  the  one  QMDO;  commonality 
of  technical  work  across  technical  approaches,  materiel  concepts,  or 
QMDOs  is  ignored.  This  is  a  rather  serious  shortcoming  since  any 
commonality  of  technology  objectives  across  the  components  of  the 
planning  structure  would  mean  that  the  model  is  double  counting  costs, 
resources,  and  "tasks”  that  would  be  involved  in  an  Exploratory  Devel¬ 
opment  program  for  a  given  profile  of  QMDOs. 

In  the  Hercules  method,  work  in  the  technologies  is  coupled  pre¬ 
cisely  to  specific  projects  and  their  products.  The  end-product 
orientation  is  not  broken  down  in  a  subsequent  procedure  for  evaluat¬ 
ing  each  technology.  Consequently,  the  extent  of  the  work  on  any 
particular  technology  has  a  clearly  defined  impact  on  the  expected 
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payoff  of  a  project.  This  provides  ready  surveillance  of  whether 
development  is  planned  in  all  the  pertinent  technologies  supporting 
a  project.  As  a  result,  however,  the  method  treats  a  technology 
either  as  a  strict  requirement  of  a  project  or  as  a  perfect  substitute 
for  another  technology  in  the  project.  As  argued  already  this  is  a 
much  too  simplistic  description  of  the  trade-offs  that  should  be 
possible.  It  does  not  make  provision  for  the  intermediate  case,  pos¬ 
sibly  more  prevalent,  in  which  the  extent  of  development  in  one  tech¬ 
nology  may  be  substituted  to  some  degree  for  development  in  another 
technology,  while  both  must  be  incorporated  to  some  extent  in  the 
project . 

As  part  of  the  Army  Missile  Plan,  technical  laboratory  personnel 
enumerate  the  technological  composition  of  the  different  systems  without 
analysis  of  any  possible  trade-offs  or  other  relationships  among  the 
technologies.  Explicit  couplings  between  systems  and  technologies 
are  emphasized  without  pretense  to  more  refined  investigation  of  the 
results  that  might  be  expected  from  variations  in  the  technological 
effort. 

c.  Conclusions.  Except  for  the  rather  rudimentary  step  incor¬ 
porated  into  the  CAL  method,  little  has  been  done  in  the  reviewed 
methods  to  give  adequate  treatment  to  the  possible  substitution  and 
complementary  relationships  of  the  various  technologies  composing 
the  different  weapon  systems.  In  some  of  the  methods,  particular 
technologies  are  related  to  the  system  by  some  measure  of  criticality. 
Presumably  technologies  which  are  more  critical  are  more  important 
to  the  system.  It  is  not  made  clear  how  this  criticality  is  measured; 
further  study  of  this  matter  is  in  order.  The  most  likely  specifica¬ 
tion  for  criticality  would  seem  to  be:  potential  advances  in  this 
technology  would  improve  in  system  effectiveness  more  than  the  poten¬ 
tial  advances  in  other  technologies.  Further,  the  achievement  of  the 
stated  level  of  advance  for  this  technology  is  more  important  to  the 
system  than  other  technologies  in  that  failure  to  meet  the  goals  pro¬ 
duces  a  greater  loss  in  the  effectiveness  of  the  system. 
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In  no  case  has  a  system  postulated  for  a  quantitative  method 
been  introduced  with  more  than  a  single  level  of  a  subsystem  quality. 

In  the  preliminary  system  analysis,  comparisons  should  have  been  made 
among  systems  using  alternative  levels  of  the  technology  in  question 
and  a  proper  decision  made  on  the  utilization  of  the  technology  level 
specified.  In  fact,  this  is  seldom  done;  the  analysis  is  much  less 
rigorous  and  based  upon  many  previous  small  studies  by  the  system 
designer. 

Without  more  intensive  analysis,  the  simplistic  enumeration  of  the 
technological  composition  of  different  weapon  systems,  such  as  used  in 
the  Army  Missile  Plan,  remains  attractive.  When  cross -classified,  a 
simple  enumeration  would  at  least  make  visible  the  technologies  used 
in  each  weapon  and  all  the  proposed  weapons  incorporating  a  particular 
technology.  In  turn,  this  visibility  could  help  the  program  planners 
understand  better  the  impacts  of  their  decisions  on  the  effort  to  be 
expended  on  a  particular  technology. 

5.  Development  Tasks 

a.  Discussion.  For  budget  allocation  purposes,  the  development 
task  is  the  basic  unit  of  effort  that  the  analyst  considers  in  plan¬ 
ning  work  to  advance  a  particular  area  of  technology.  A  task  is  fairly 
specific  in  terms  of  the  technological  goals  it  attempts  to  achieve 
and  the  procedure  it  follows.  As  a  factor  that  should  be  considered 
in  a  quantitative  method  for  allocating  the  development  budget,  a  task 
has  two  interesting  aspects.  One  of  these  aspects  is  the  pattern  of 
developmental  inputs  that  is  used  in  the  task.  The  other  aspect  is  the 
position  of  the  particular  task  as  a  component  of  a  broader  develop¬ 
ment  effort  directed  at  more  complex  technological  accomplishments. 

In  the  first  of  these  aspects,  the  task  is  broken  down  into  de¬ 
scriptions  of  how  different  patterns  of  development  inputs  may  be 
related  to  steps  in  the  advance  of  the  technology.  These  relationships 
merge  organizational  and  behavioral  factors  to  derive  alternative  com¬ 
binations  of  development  personnel,  materials,  equipment,  facilities 
and  other  inputs  that  might  be  used  to  advance  the  knowledge  about  and 
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application  of  a  technology.  On  the  basis  of  such  relationships, 
management  can  better  understand  how  the  inputs  can  substitute  for 
each  other  and  how  they  may  complement  each  other  in  a  development 
effort. 

The  second  aspect  sets  the  limits  of  the  development  context 
within  which  the  particular  task  is  performed  and  includes  the  frame¬ 
work  within  which  the  analyst  can  formulate  relationships  among  the 
tasks  to  spell  out  alternative  development  strategies.  In  this  aspect, 
the  analyst  is  concerned  with  the  sequencing  of  the  various  develop¬ 
ment  tasks,  the  extent  to  which  their  results  are  interdependent,  and 
consequently  the  coordination  required  among  them. 

The  results  of  some  development  tasks  may  be  completely  inde¬ 
pendent  of  the  results  of  others  within  the  particular  context.  Such 
tasks  can  be  performed  either  in  parallel  or  in  sequence,  depending 
upon  other  considerations.  On  the  other  hand,  the  results  of  some 
other  tasks  might  be  highly  interdependent  in  a  number  of  ways.  First, 
the  overall  development  goal  may  require  fairly  tight  combination  of 
the  outputs  of  the  tasks.  Second,  the  results  of  some  tasks  might  be 
necessary  inputs  to  other  tasks.  Third,  some  tasks  may  cover  substi¬ 
tutable  technological  goals  so  that  proceeding  with  a  particular  group 
of  them  may  depend  upon  whether  another  group  succeeds  in  achieving 
its  goal.  Analysis  of  the  extent  to  which  such  tasks  can  be  carried 
out  sequentially  or  in  parallel  provides  information  describing,  among 
other  things,  the  development  strategies  that  will  result  from  differ¬ 
ent  allocations  of  resources  among  the  tasks. 

b.  Development  Tasks  in  the  Reviewed  Quantitative  Methods.  With 
the  exception  of  the  Industrial  Analog  method,  all  of  the  quantitative 
methods  reviewed  break  down  the  work  that  is  to  be  done  in  each  tech¬ 
nology  into  a  set  of  development  tasks.  The  Industrial  Analog  attempts 
to  determine  the  level  of  effort  that  should  be  devoted  to  a  particular 
technology  without  specifying  precise  technical  goals  for  separable 
units  of  work. 
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Among  the  other  methods,  the  treatment  of  the  development  task  as 
an  element  in  Exploratory  Development  planning  is  highly  diverse.  The 
formulators  of  the  methods  not  only  resort  more  or  less  explicitly  to 
tasks  as  the  basic  unit  of  effort,  they  also  analyze  the  composition  of 
the  tasks’  resource  requirements  in  varying  detail  and  spell  out  the 
interdependencies  among  tasks  only  to  a  modest  extent. 

In  the  NOL  method,  tasks  are  not  included  explicitly  as  planning 
factors,  but  some  concept  of  them  must  be  behind  the  functional  rela¬ 
tionships  that  describe  the  different  levels  of  the  technical  pacing 
parameter  that  can  be  achieved  with  different  efforts. 

Other  methods,  such  as  TORQUE,  FDL,  CAL,  Hercules,  and  the  Army 
Missile  Plan,  do  set  out  specific  technical  goals  that  each  separable 
development  task  would  attempt  to  fulfill.  In  some  cases,  like  TORQUE, 
these  tasks  are  also  sequenced  if  they  are  considered  to  fit  together, 
either  side  by  side  or  end  on  end,  in  fulfilling  somewhat  higher  and 
more  complex  technical  goals. 

However,  the  methods  do  not  generally  analyze  explicitly  the  re¬ 
source  composition  of  the  tasks  or  alternative  patterns  of  resources 
in  a  specific  task  (or  alternative  tasks)  that  might  achieve  the  same 
basic  technical  goal. 

The  FDL  and  CAL  methods  are  exceptions.  The  FDL  efforts  on  these 
matters  are  not  extensive,  however.  The  FDL  method  records  only  the 
in-house  and  contract  engineering  manpower  composition  of  each  task 
but  does  so  in  a  way  that  permits  these  two  inputs  to  substitute  for 
or  complement  each  other  in  a  variety  of  patterns. 

In  the  CAL  method,  provision  is  made  for  explicit  treatment  of 
the  alternative  patterns  of  inputs  that  might  be  used  in  tasks  that 
are  directly  interchangeable  for  each  other.  The  Army  Materiel  Com¬ 
mand  R&D  Field  Establishments  and  other  resources  make  up  the  inputs 
that  are  accounted  for  in  this  way. 

Among  the  methods  reviewed,  the  CAL  method  is  practically  unique 
in  setting  out  explicitly  alternative  sets  of  tasks  that  might  be 
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followed  to  fulfill  a  particular  materiel  concept.  However,  this 
approach  only  partially  opens  the  question  of  choosing  a  development 
strategy.  More  than  one  set  of  tasks  (technical  approaches)  can  be 
adopted  to  ensure  the  success  of  the  materiel  concept.  However,  the 
method  treats  those  sets  as  though  they  are  executed  in  parallel.  It 
does  not  make  provision  for  following  the  different  sets  of  tasks 
sequentially. 

c.  Conclusions.  In  general,  the  methods  reviewed  treat  somewhat 
cursorily  composition,  context,  and  interrelationships  of  the  develop¬ 
ment  tasks  into  which  they  break  up  their  Exploratory  Development  pro¬ 
grams.  Possibly,  this  results  from  difficulty  with  deriving  precisely 
the  technical  goals  that  should  be  fulfilled  from  consideration  of 
general,  not  specific,  weapon  system  concepts.  If  this  difficulty 
characterizes  planning  an  Exploratory  Development  program  and  allo¬ 
cating  the  budget  within  that  program,  then  some  very  basic  conditions 
of  the  problem  would  appear  to  be  fairly  inconsistent  with  the  appli¬ 
cation  of  highly  precise  quantitative  methods  to  the  allocation  of  the 
budget.  On  the  other  hand,  inadequate  treatment  of  alternative  tasks 
that  could  fulfill  specific  technical  goals  and  the  wide  range  of  re¬ 
lationships  that  can  exist  among  different  tasks  also  appear  to  be  in¬ 
consistent  with  the  use  of  precise  budget  allocation  methods. 

Unless  alternative  tasks  and  the  relationships  among  tasks  can  be 
treated  more  completely  and  precisely,  simpler  methods  for  deriving 
and  recording  the  development  tasks  of  an  Exploratory  Development  pro¬ 
gram  are  probably  more  desirable.  For  example,  in  the  Army  Missile 
Plan,  the  technical  laboratories  prepare  a  plan  of  tasks  to  achieve 
the  technical  advances  required  for  the  designs  of  the  proposed  weapons. 
The  linkage  of  the  tasks  to  the  weapons  is  explicit  and  remains  so 
throughout  the  subsequent  steps  of  the  Exploratory  Development  program 
planning  process. 

6.  Costs 

a.  Discussion.  The  development  task  is  the  administrative 
planning  unit  for  programming  the  specific  application  of  development 


119 


resources  (such  as  manpower,  material,  and  facilities).  Therefore,  it 
provides  the  background  for  estimating  the  costs  that  will  be  incurred 
in  the  development  process.  The  estimated  costs  of  a  development  pro¬ 
gram  should  measure  the  total  expenditures  management  must  make  to  ob¬ 
tain  the  resources  needed  for  the  execution  of  the  program. 

Three  aspects  of  development  costs  are  interesting  for  purposes 
of  allocating  the  development  budget.  One  of  these  aspects  is  the 
level  of  detail  at  which  the  costs  are  taken  into  account.  A  second 
aspect  is  the  scope  of  the  costs  that  should  be  taken  into  account. 

The  third  aspect  is  when  the  costs  must  be  incurred  to  carry  out  the 
planned  operations  of  the  tasks. 

Depending  upon  the  level  of  detail  desired  in  the  accounting, 
the  tally  can  be  done  in  terms  of  general  classes  of  resources  or 
broken  down  into  finer,  more  specific  inputs.  Among  the  general 
classes  of  resources,  the  analyst  might  include  materials,  operating 
inputs  (such  as  labor  and  services),  and  facilities.  Even  with  such 
a  broad  breakdown,  explicit  account  should  be  taken  of  new  investment 
requirements . 

With  finer  detail,  the  analyst  could  focus  on  the  specific  capa¬ 
bilities  of  the  different  groups  of  research -engineering  personnel  and 
the  applications  of  specialized  equipment. 

Regardless  of  the  level  of  detail  desired,  the  scope  of  the  costs 
should  include  all  expenditures  for  all  resources  employed  in  the  de¬ 
velopment  task.  This  may  not  be  a  simple  and  straightforward  account¬ 
ing  exercise.  Depending  upon  the  budgeting  techniques  followed,  some 
resources  actually  employed  in  the  development  process  may  be  financed 
from  other  programs.  For  example,  compensation  for  military  personnel 
in  development  activities  is  not  included  in  the  appropriation  accounts 
for  military  RDT&E.  Expenditures  that  are  directly  related  to  the 
level  of  the  development  effort  but  made  through  other  appropriations 
should  be  included  within  the  scope  of  the  total  development  costs, 
possibly  treated  as  transfers. 
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Moreover,  some  real  costs  are  frequently  ignored  completely  in 
Government  accounting  techniques.  For  example,  the  user  and  interest 
costs  that  should  be  associated  with  the  employment  of  such  resources 
as  land  are  frequently  overlooked.  If  a  particular  Department  or  the 
Government  owns  the  land,  it  generally  does  not  impute  such  costs  to 
its  current  operations  since  it  does  not  make  corresponding  expendi¬ 
tures  even  though  those  resources  could  command  compensation  from 
employment  elsewhere . 

Another  important  consideration  is  the  timing  of  the  expenditures 
made  to  obtain  the  resources  employed  over  the  life  of  the  development 
task.  Recording  the  total  costs  and  when  they  will  be  expended  gives 
the  manager  the  opportunity  to  evaluate  the  impacts  of  different  time 
patterns  of  expenditure,  possibly  for  budgeting  purposes,  and  to  have 
all  the  costs  over  the  life  of  the  task  visible  at  one  time.  The 
latter  should  help  to  prevent  wedging  in  high-cost  development  tasks 
through  the  illusion  of  low  first -year  costs.  Keeping  track  of  when 
the  expenditures  must  be  made  also  helps  to  determine  the  relevant 
scope  of  costs  that  should  be  considered.  Expenditures  actually 
made  before  the  inception  of  the  investment  activities  and  operations 
of  a  particular  task  are  not  pertinent  to  the  measure  of  costs  that 
must  be  incurred  for  its  execution.  These  expenditures  have  already 
been  made,  no  further  Treasury  disbursements  must  be  made  to  permit 
use  of  the  associated  inputs,  and  those  inputs  have  no  current  alter¬ 
native  application. 

b.  Costs  in  the  Reviewed  Quantitative  Methods.  With  the  excep¬ 
tion  of  the  Industrial  Analog  method,  all  of  the  methods  reviewed  tally 
costs  and  resource  requirements  as  a  part  of  planning  the  component 
tasks  of  the  development  program.  The  Industrial  Analog  tries  to 
compute  the  expenditures  that  should  be  made  at  a  more  aggregative 
level  covering  a  broad  technological  area. 

Most  of  the  methods  provide  for  estimating  the  costs  of  the 
tasks  in  the  broadest  aggregates,  usually  only  the  total  costs  that 
will  be  incurred  in  the  execution  of  the  task.  The  CAL  method, 
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however,  does  provide  for  recording  the  expenditures  made  on  the  vari¬ 
ous  resources  employed  in  the  different  tasks.  Of  course,  the  main 
virtues  of  estimating  task  costs  in  some  detail  are  the  explicitness 
the  detail  requires  in  the  calculations  and  the  possibilities  for 
verification  contained  in  the  detailed  records. 

None  of  the  methods  reviewed  describe  in  any  detail  the  specific 
costs  that  should  be  tallied  for  consideration  in  the  budget  alloca¬ 
tion  process.  This  probably  means  that  all  of  them  let  the  analyst 
making  the  specific  application  decide  what  resource  costs  should  be 
included.  Or  it  might  mean  that  the  formulators  were  not  aware  of  or 
concerned  with  setting  the  limits  on  the  scope  of  the  costs  that  are 
relevant  for  determining  the  composition  of  an  Exploratory  Develop¬ 
ment  program. 

The  treatment  of  the  timing  of  the  expenditures  that  must  be 
made  for  each  development  task  is  quite  diverse  across  the  methods 
reviewed,  ranging  from  ignoring  any  costs  that  would  be  expended  in 
any  time  beyond  the  current  budget  period  to  attempting  an  accounting 
of  the  total  costs  required  over  the  task  life.  Although  methods  such 
as  NOL,  FDL,  and  CAL  make  provisions  for  different  levels  of  expendi¬ 
tures,  they  appear  to  pay  little  if  any  attention  to  the  costs  that 
must  be  incurred  during  the  life  of  the  task  beyond  the  specific 
budget  period  of  immediate  interest.  The  Hercules  method  attempts  to 
consider  the  total  costs  that  will  be  expended  and  the  pattern  of  those 
expenditures  for  each  project  eligible  for  funding.  In  both  TORQUE 
and  the  Army  Missile  Plan,  the  technical  personnel  who  devise  the 
development  tasks  that  must  be  performed  also  estimate  the  pattern  of 
the  total  costs  to  be  expended  over  the  lives  of  the  individual  tasks. 
The  Missile  Plan  carries  the  time  pattern  of  expenditures  through  its 
subsequent  steps  but  pays  most  attention  to  the  current  year's  budget 
and  the  corresponding  expenditures  in  each  task.  TORQUE  does  try  to 
combine  considerations  for  both  the  expenditures  to  be  made  on  the 
development  task  in  the  current  budget  year  and  the  total  costs  to  be 
expended  upon  the  task  over  the  remainder  of  its  duration. 
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Because  the  cost  concepts  employed  in  the  Industrial  Analog 
method  exemplify  problems  of  detail,  scope  and  timing,  they  warrant 
more  detailed  attention.  The  Industrial  Analog  method  proposes  to 
use  as  an  indicator  of  the  desired  budget  allocation  to  a  technical 
area  of  military  exploratory  development  the  ratio  of  company-funded 
R&D  expenditures  to  net  industry  sales. 

By  taking  into  account  only  the  company-funded  R&D  expenditures 
by  American  industry,  the  measure  ignores  the  substantial  amount  of 
research  and  development  work  that  is  supported  by  the  Federal  Govern¬ 
ment  in  these  industries.  The  incidence  of  this  support  is  by  no 
means  uniform  as  shown  in  Table  5. 

Federal  and  company-funded  expenditures  on  R&D  within  an  industry 
are  complementary,  both  contributing  increased  information  about  the 
technical  field  of  study.  Inasmuch  as  the  companies  performing  the 
research  and  development  generally  can  make  commercial  applications 
of  the  information  generated  by  either  funding  source,  Federal  funding 
may  be  a  significant  contributor  to  their  commercial  success.  If 
this  is  so,  an  Industrial  Analog  for  Defense  R&D  based  only  upon 
company-funded  R&D  expenditures  could  substantially  undercount  the 
inputs  that  have  been  necessary  for  realized  success. 

Of  course,  counting  the  Federally  funded  industry  R&D  effort  only 
leads  to  a  further  problem.  If  the  Federal  funds  were  included  in  the 
ratio  and  the  ratio  were  an  important  component  of  the  Defense  alloca¬ 
tion  method,  Defense  R&D  funding  would  not  be  independent  of  itself. 

In  other  words,  the  allocation  process  would  be  a  circular  one,  turn¬ 
ing  back  into  itself,  and  ultimately  self -confirming. 

R&D  funding  data  reported  on  a  company  basis  must  be  used  with- 
care,  especially  if  the  company  data  are  to  be  used  to  measure  the  R&D 
effort  expended  in  the  different  product  categories.  For  some  report¬ 
ing  purposes,  companies  are  classified  in  an  industry  according  to 
their  primary  product,  any  single  company  being  in  only  one  industry 
at  any  one  time.  Insofar  as  companies  have  substantial  interests  in 
secondary  products,  counting  all  of  their  R&D  funding  as  applying  to 
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TABLE  5.  FUNDS  FOR  R&D  PERFORMANCE,  BY  INDUSTRY,  1965 


Industry 

Total 

Company 

Funds 

Federal 

Funds 

Federal  Funds 
as  Percent  of 
total  R&D  Funds 

(Millions  of  Dollars) 

Total - 

$14,197 

$6,438 

$7,759 

55 

Food  and  kindred  products - 

150 

148 

1 

'  1 

Textiles  and  apparel - - - 

34 

(b) 

(a) 

(a) 

Lumber,  wood  products,  and  furniture - 

13 

(b) 

(a) 

(a) 

Paper  and  allied  products - 

76 

76 

-- 

— 

Chemicals  and  allied  products - 

1,377 

1,187 

190 

14 

Industrial  chemicals - 

928 

781 

mm 

16 

Drugs  and  medicines - 

268 

(b) 

(a) 

Other  chemicals - 

181 

(b) 

£1H 

(a) 

Petroleum  refining  and  extraction - 

435 

366 

69 

16 

Rubber  products - 

166 

141 

25 

15 

Stone,  clay  and  glass  products - 

119 

115 

4 

3 

Primary  metals - 

216 

208 

8 

4 

Primary  ferrous  products - 

131 

130 

1 

1 

Nonferrous  and  other  metal  products - 

85 

79 

7 

8 

Fabricated  metal  products - 

145 

129 

17 

11 

Machinery - 

1,129 

870 

258 

23 

Electrical  equipment  and 

communications - 

3,167 

1,189 

1,978 

62 

Communications  equipment  and 

electronic  components - 

1,912 

659 

1,253 

66 

Other  electrical  equipment - 

1,255 

530 

72  5 

58 

Motor  vehicles  and  other  transportation 

equipment - 

1,238 

913 

326 

26 

Aircraft  and  missiles - 

5,120 

620 

4,500 

88 

Professional  and  scientific  instruments-- 

387 

261 

12  5 

32 

Scientific  and  mechanical  measuring 

instruments - 

76 

58 

18 

24 

Optical,  surgical,  photographic  and 

other  instruments - 

311 

203 

107 

35 

Other  manufacturing  industries - 

67 

66 

1 

2 

Nonmanufacturing  industries - 

359 

103 

255 

71 

Source:  Ref.  20. 


aNot  separately  available  but  included  in  total. 
^Less  than  $0.5  million. 
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the  products  of  the  industry  in  which  they  are  classified  may  be  mis¬ 
leading  for  determining  the  distribution  of  R&D  effort  among  technical 
fields  or  products.  The  data  in  Table  6  illustrate  this  problem. 

For  example,  of  a  $3.0  billion  total,  the  electrical  equipment  and 
communications  industry  spent  approximately  $0.73  billion  on  applied 
research  and  development  of  atomic  energy  devices  and  guided  missiles 
and  spacecraft. 

Three  sources  have  been  identified  for  the  interindustry  differ¬ 
ences  in  the  ratios  of  company-funded  R&D  expenditures  to  net  sales 
(Ref.  21,  pp.  59,  61).  First,  the  demand  for  the  increased  perform¬ 
ance  of  newly  developed  products  is  more  or  less  intense  in  the  various 
industries.  Second,  the  cost  or  ease  of  generating  significant  new 
products  is  different  by  industry.  Third,  the  amount  of  R&D  performed 
in  an  industry  tends  to  vary  with  the  size  distribution  of  the  com¬ 
panies  making  up  the  industry,  industries  with  predominantly  very 
small  companies  spending  relatively  less  than  those  industries  com¬ 
posed  mainly  of  larger  companies. 

While  the  demand  and  cost  conditions  of  the  first  two  sources  of 
the  interindustry  differences  are  quite  relevant  and  desirable  guides 
for  directing  company-funded  R&D  effort,  the  appropriateness  of  the 
first  for  determining  Defense  R&D  remains  questionable.  The  demand 
for  military  goods  may  well  be  a  component  of  the  demand  for  the 
higher  performance  products  from  an  industry,  but  the  total  demand 
for  the  higher  performance  products  is  also  affected  substantially 
by  private  sector  and  other  Government  preferences.  That  the  total 
demand  should  correspond  in  some  way  to  the  demand  for  Defense  per¬ 
formance  improvements  is  a  strong  condition  that  would  hold  only 
accidentally. 

The  third  source  of  the  interindustry  differences,  the  company- 
size  distribution  of  the  industries,  is  totally  irrelevant  for  estab¬ 
lishing  Defense  R&D  priorities.  In  fact,  this  source  could  well 
generate  undesirable  results.  In  technical  fields  for  which  the 
principal  industry  is  composed  primarily  of  very  small  firms,  the 
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ratio  will  be  relatively  lower,  indicating  less  DoD  R&D  funding  than 
might  be  supported  by  other  economic  criteria,  especially  if  the  R&D 
benefits  are  dispersed.  On  the  other  hand,  in  technical  fields  for 
which  the  principal  industry  is  composed  of  large  firms,  the  ratio 
will  be  relatively  higher.  This  could  result  in  a  higher  level  of  DoD 
R&D  funding  for  those  fields  than  would  serve  defense  efficiently, 
especially  if  the  company-funded  R&D  is  essentially  dissipated  on 
product  differentiation  to  increase  or  maintain  company  market  power. 

Moreover,  the  ratio  of  company-funded  R&D  expenditures  to  net 
sales  is  an  after-the-fact  description  of  industry  behavior,  not  a 
prescription  of  how  industry  should  or  is  going  to  act.  Even  if 
industry  had  some  infallible  technique  for  determining  its  R&D  budget 
size  and  allocation  that  was  also  appropriate  for  Defense,  the  ratio 
description  of  past  behavior  would  give  direction  that  could  lag 
substantially  behind  the  "desired’’-  path. 

Mansfield’s  investigations  (Ref.  21,  pp.  62-63)  indicate  that, 
in  the  short  run,  companies  tend  to  maintain  a  fairly  constant  ratio 
of  R&D  expenditures  to  sales.  However,  given  a  longer  time  to  adjust, 
companies  change  their  desired  ratio.  The  desired  ratio  generally 
increases  with  (1)  increases  in  the  forecasted  profitability  of  R&D 
products,  (2)  "bandwagon  effects"  created  by  increases  in  R&D  in  re¬ 
lated  segments  of  industry,  (3)  decreases  in  the  projected  profita¬ 
bilities  of  alternative  uses  of  available  funds,  and  (4)  the  tendency 
to  maintain  an  absolute  level  of  effort  in  the  face  of  declining 
sales.  A  company  may  decrease  the  ratio  as  sales  expand  quickly  in 
order  to  avoid  the  costs  of  rapid  R&D  expansion  by  building  up  to  a 
desired  level  over  a  period  of  years.  The  speed  with  which  a  company 
adjusts  its  actual  ratio  to  the  desired  ratio  appears  to  be  influenced 
by  (1)  the  difference  between  the  desired  ratio  and  the  previous  year’s 
ratio  and  (2)  the  ratio  of  the  previous  year’s  R&D  expenditures  to 
profits . 

c.  Conclusions .  The  level  of  detail  at  which  the  costs  of  exe¬ 
cuting  Exploratory  Development  tasks  are  tallied  is  not  crucial  to  the 
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TABLE  6.  FUNDS  FOR  APPLIED  RESEARCH  AND  DEVELOPMENT  PERFORMANCE 
BY  INDUSTRY  AND  PRODUCT  FIELD,  1965 

(Millions  of  dollars) 


r>- 

04 

rH 


budget  allocation  procedure.  Detail  may  force  explicitness  in  the 
calculations  and  permit  more  ready  verification  of  the  cost  estimates; 
however,  it  can  be  neutral  in  generating  a  desirable  allocation  of 
development  funds. 

However,  determining  the  proper  scope  of  the  costs  that  should  be 
taken  into  account  and  the  timing  of  the  incidence  of  those  costs  are 
extremely  important  considerations  for  establishing  a  desirable  allo¬ 
cation  of  the  development  budget  among  technologies.  Ignoring  the 
question  of  the  scope  of  the  costs  that  should  be  included  among  the 
factors  considered  in  a  quantitative  method  is  quite  similar  to  act¬ 
ing  like  an  ostrich.  Without  accounting  for  the  proper  scope  of  costs 
and  the  timing  of  their  incidence,  the  Exploratory  Development  manager 
could  well  choose  a  set  of  development  tasks  that  imposes  substantially 
higher  costs  on  the  Defense  community  or  the  economy  as  a  whole  than 
would  be  necessary  to  accomplish  the  same  technical  goals.  Costs  in¬ 
curred  for  the  execution  of  the  task  but  financed  from  another  appro¬ 
priation  or  in  the  future  are  nonetheless  real  and  burdensome  on  the 
productive  capacity  of  the  economy. 

The  methods  reviewed  have  adequately  treated  neither  the  scope 
of  the  costs  nor  their  timing.  None  raise  the  question  of,  let  alone 
analyze,  the  scope  of  the  costs  to  be  included  among  their  considera¬ 
tions  . 

Because  of  an  inordinate  concern  for  the  one -year  budgeting  pro¬ 
cedure,  most  of  the  methods  reviewed  consider  only  the  costs  that 
would  be  incurred  by  a  project  or  program  in  the  next  year.  If  it 
accomplishes  anything,  such  an  approach  puts  a  heavy  premium  on  getting 
very  large  projects  in  the  door  by  starting  them  at  very  low  levels  of 
effort. 

Concern  for  total  project  life  costs  does  not  require  neglecting 
the  budgetary  procedure.  In  fact,  a  better  picture  of  future  budget 
needs  can  be  produced  using  total  costs  and  dating  expenditures  ac¬ 
cording  to  the  time  they  will  be  made. 
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7 .  Timing 


a.  Discussion.  The  timing  of  the  events  in  a  development  pro¬ 
gram  and  the  amount  of  time  that  elapses  in  the  course  of  those  events 
are  important  factors  that  should  be  considered  in  planning  an  Explora¬ 
tory  Development  program  and  allocating  the  programTs  budget.  For  the 
most  part,  these  timing  factors  characterize  some  aspects  of  the 
factors  already  analyzed.  Reference  has  been  made  to  these  aspects 
in  several  instances  above.  However,  where  timing  could  be  separated 
from  that  discussion  without  giving  misleading  impressions,  the  analysis 
of  timing  was  postponed. 

Timing  is  a  thread  running  thoughout  the  range  of  factors.  When 
an  operational  requirement  is  fulfilled,  when  a  weapon  system  is  com¬ 
pleted,  when  a  crucial  technology  is  advanced  to  a  useful  level,  when 
development  tasks  are  undertaken  and  completed,  and  when  expenditures 
must  be  made  to  obtain  the  inputs  necessary  for  a  development  task 
are  all  important  features  of  the  individual  factors.  In  combination, 
these  features  require  coordination  and  integration  if  the  development 
program  is  to  be  successful  and  ultimately  contribute  to  higher  order 
Defense  objectives. 

For  purposes  of  Exploratory  Development  program  planning,  the 
timing  of  the  fulfillment  of  an  operational  requirement  and  the  com¬ 
pletion  of  a  weapon  system  are  principally  external  considerations 
derived  from  other  Defense  criteria.  However,  the  timing  of  the 
advance  of  a  technology  and  the  timing  of  a  development  task  fall, 
to  a  large  extent,  within  the  proper  scope  of  Exploratory  Development 
planning.  The  basic  considerations  are  the  time  that  is  required  to 
complete  a  development  task  and  the  timing  of  that  development  task 
relative  to  other  tasks  with  which  it  might  be  interdependent.  These 
determine  the  timing  of  any  more  complex  technological  advances. 

Time-cost  trade-offs  arise  when  the  time  elapsed  in  the  execution 
of  a  particular  development  task  can  be  varied  or  when  tasks  with 
essentially  the  same  technical  objective  have  different  time  require¬ 
ments.  Time -performance  trade-offs  can  also  be  made  in  a  similar 
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manner  if  tasks  having  somewhat  different  technical  objectives  and 
time  requirements  are  essentially  substitutable  for  each  other. 

The  development  strategy  followed  in  any  program  consists,  at 
least  in  the  first  instance,  of  determining  the  timing  of  the  execution 
of  the  individual  tasks  within  a  group  that  is  interdependent.  A 
strategy  of  concurrency  is  followed  when  a  number  of  interrelated  tasks 
are  undertaken  at  the  same  time.  A  sequential  strategy  is  followed 
when  the  timing  of  the  interrelated  tasks  is  arranged  so  that  only  one 
is  executed  at  a  particular  time  with  the  execution  of  the  others 
being  dependent  upon  its  success  or  failure. 

Timing  of  the  expenditures  that  must  be  made  to  obtain  the  re¬ 
sources  employed  in  the  development  tasks  has  been  discussed  in  some 
detail  in  Section  IV-C-6. 

Taking  into  account  the  timing  aspects  of  all  the  other  factors 
in  a  method  for  allocating  the  Exploratory  Development  budget  is 
obviously  important  for  management.  It  not  only  helps  management  to 
know  when  technological  developments  or  weapon  systems  will  be  avail¬ 
able,  it  also  gives  management  the  information  that  it  can  use  to 
exercise  whatever  discretion  it  may  have  to  influence  the  timing  of 
individual  tasks,  sets  of  tasks,  and  expenditures.  Management  can 
apply  its  time  preferences  and  discount  rates  to  the  time-phased 
technological  advances  and  expenditures  to  compare  alternative  courses 
of  action  and  choose  among  them. 

b.  Timing  in  the  Reviewed  Quantitative  Methods.  Among  the  meth¬ 
ods  reviewed,  the  Industrial  Analog  does  not  take  into  account  any  con¬ 
sideration  of  the  timing  of  the  development  effort  or  the  expenditures 
for  the  inputs  employed.  As  was  pointed  out  above,  its  lack  of  ex¬ 
plicit  treatment  of  this  matter  results  in  it  proposing  to  relate 
current  and  future  Defense  expenditures  on  a  technical  area  to  past 
commercial  expenditures  in  related  fields. 

Most  of  the  other  methods  reviewed  have  two  common  character¬ 
istics  in  their  treatment  of  timing.  They  generally  approach  the 
problem  of  allocating  the  Exploratory  Development  budget  as  though  it 
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extends  through  one  time  period,  with  funds  and  work  to  be  chosen  only 
for  the  next  budget  period.  Also,  the  methods  pay  little  attention  to 
variations  that  are  possible  in  the  completion  times  of  the  different 
development  tasks. 

Generally,  a  fair  amount  of  timing  information  is  behind  the  treat 
ment  given  the  various  factors  in  the  different  methods.  Some  of  it  is 
extensive  and  used  quite  explicitly  to  put  the  data  on  the  factors  in 
the  form  incorporated  into  the  quantitative  operations  of  the  methods. 
In  the  NOL  method,  however,  timing  considerations  are  implicit,  at 
best.  It  is  principally  concerned  with  the  extent  of  the  advance 
that  can  be  made  in  the  pacing  parameters  over  the  next  budget  year 
without  explicit  reference  to  any  target  times  for  a  level  of  advance, 
weapons,  or  EDGs. 

The  Hercules  method  considers  explicitly  (1)  the  timing  of  the 
marketing  of  the  product  from  the  development  project,  (2)  the  time 
required  to  achieve  the  technical  goals  needed  to  complete  the  project, 
and  (3)  the  modification  of  the  project  completion  time  that  is  pos¬ 
sible  by  varying  expenditures  on  component  tasks.  However,  the  method 
does  not  take  into  account  the  time  preferences  that  management  may 
have  for  the  sales  and  profits  of  the  project’s  product  or  for  the 
expenditures  that  must  be  made  to  complete  the  project.  Applying  a 
discount  factor  to  profits  and  expenditures  could  easily  remedy  this 
last  shortcoming. 

Because  of  the  flexibility  it  has  in  the  specification  of  differ¬ 
ent  technical  approaches  that  can  be  followed  to  a  particular  materiel 
concept,  the  CAL  method  can  similarly  consider  the  timing  of  the  com¬ 
ponent  development  tasks  and  the  satisfaction  of  each  QMDO. 

The  TORQUE,  FDL,  and  Army  Missile  Plan  methods  utilize  detailed 
information  on  the  target  times  of  the  weapon  systems  their  develop¬ 
ment  programs  should  support,  the  timing  of  the  development  tasks 
making  up  the  programs,  and  the  timing  of  the  expenditures  that  must 
be  made  for  alternative  tasks.  The  Army  Missile  Plan  carries  this 
information  along  in  the  budget  allocation  process,  being  concerned 
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primarily  with  the  time  at  which  the  weapon  system  will  be  available 
and  the  tasks  that  should  be  performed  in  the  next  budget  period. 
TORQUE  and  the  FDL  method  use  the  timing  data  in  more  elaborate  cal¬ 
culations  for  evaluating  the  level  and  timing  of  the  development  tasks 
that  should  be  executed. 

Both  TORQUE  and  the  FDL  methods  calculate  factors  that  are  used 
to  set  the  value  of  each  development  task  according  to  whether  the 
timing  of  its  successful  completion  will  permit  completion  of  the 
weapon  system  incorporating  it  by  the  weapon's  target  date.  Besides 
the  completion  date  of  the  task  and  the  target  dates  of  the  weapon 
system,  the  pivotal  concept  used  in  this  calculation  is  the  timeliness 
function  (Fig.  2). 

...Two  [target]  dates  are  needed  for  each  system/subsystem 
to  reflect  the  time  span  during  which  initial  deployment 
in  operational  units  is  desired  for  maximum  overall  effec¬ 
tiveness.  The  'earliest  date'  needed  is  such  that  existing 
equipment,  or  equipment  scheduled  for  procurement,  is  ex¬ 
pected  to  still  be  satisfactory.  The  'latest  date'  recog¬ 
nizes  the  uncertainty  involved  in  establishing  a  precise 
initial  deployment  date.  (Ref.  8,  p.  7) 

However,  the  manuals  do  not  indicate  the  criteria  that  should  be 
applied  by  the  systems  analysts  to  judge  whether  existing  equipment  or 
equipment  scheduled  for  procurement  remains  "satisfactory. " 

The  timeliness  function  used  in  TORQUE  and  the  FDL  method  is 
based  on  the  two  valid  premises  that  (1)  development  of  a  technology 
objective  long  before  it  is  needed  in  a  weapon  system  may  be  wasteful 
of  current  resources,  foreclosing  their  use  in  projects  with  nearer 
term  payoff,  and  (2)  completion  of  a  technology  objective  for  the 
introduction  of  a  weapon  system  is  wasteful  if  its  relative  effective¬ 
ness  has  already  deteriorated  because  of  changing  external  conditions. 
Both  premises  are  indicative  of  behavior  that  should  be  avoided.  How¬ 
ever,  the  particular  timeliness  function  that  has  been  devised  is  a 
rather  rigid  proscription  of  behavior,  with  possible  mischievous 
consequences. 
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The  judgment  is  arbitrary  that  development  of  a  technology  objec¬ 
tive  before  its  ’’need  date"  is  absolutely  wasteful  (the  TORQUE  timeli¬ 
ness  function  dictates  that  a  technology  objective  developed  more 
than  two  years  before  its  "earliest  need  date"  is  worth  nothing). 

Waste  of  resources  is  better  judged  by  whether  they  could  be  employed 
in  developments  with  greater  "utility."  Of  course,  a  unit  of  utility 
achieved  sometime  in  the  future  is  not  preferred  to  a  unit  of  utility 
achieved  at  present.  However,  the  trade-off  between  present  utility 
and  future  utility  is  far  better  evaluated  through  analytical  pro¬ 
cedures  reflecting  time  preferences  rather  than  arbitrary  cut-offs 
of  the  planning  time  frame. 

With  the  timeliness  function,  the  rapid  development  of  a  tech¬ 
nology  objective  supporting  a  very  important  system  that  is  scheduled 
for  procurement  in  the  relatively  distant  future  may  receive  no  utility 
credit  because  of  the  arbitrary  cut-off.  A  result  may  well  be  that 
TORQUE  would  divert  funds  to  very  "low  priority"  efforts  because  they 
are  closer  in  time,  contrary  to  Defense  management's  time  preferences 
for  the  utility  of  the  more  remote  but  important  system. 

The  timeliness  function  is  similarly  arbitrary  in  the  way  it 
describes  and  handles  the  declining  relative  effectiveness  of  a  weap¬ 
on  system  as  the  threat  and  newer  weapon  systems  develop. 

In  general,  explicit  treatment  in  systems  analysis  of  each 
prospective  system's  effectiveness  relative  to  (1)  the  current  forces; 
(2)  changes  in  tactics,  force  structure,  and  force  level;  (3)  other 
prospective  systems;  and  (4)  the  evolution  of  the  threat,  would  be 
far  more  desirable  for  tracing  the  value  of  that  particular  system 
over  time.  The  timeliness  function  attempts  to  integrate  all  of  these 
considerations  into  a  single,  rigid,  and  arbitrary  expression,  appli¬ 
cable  to  all  systems  regardless  of  the  surrounding  circumstances. 

The  estimates  that  must  be  made  to  implement  explicit  treatment 
may  have  large  variances  because  of  the  uncertainties  of  future  weapon 
system  performance  and  the  evolving  threat.  Their  accuracy,  however, 
could  hardly  be  less  than  that  of  the  timeliness  function.  On  the 
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other  hand,  the  increased  flexibility  that  should  result  from  explicit 
and  individual  attention  to  weapon  system  effectiveness  will  ensure 
that  important  options  are  not  excluded  from  consideration. 

c.  Conclusions .  The  timing  of  the  execution  of  development 
tasks,  the  expenditures  for  resources  employed  in  the  tasks,  and  the 
achievement  of  technological  advances  cannot  be  ignored  in  determining 
the  allocation  of  the  development  budget.  These  timing  aspects  are 
important  considerations  that  must  be  taken  into  account  if  alterna¬ 
tive  development  tasks  having  similar  technical  objectives  but  differ¬ 
ent  time  requirements  can  be  selected  and  if  the  timing  of  the  tech¬ 
nological  advance  is  at  all  significant  to  management. 

Some  of  the  methods  reviewed  call  for  a  large  amount  of  detailed 
information  on  the  timing  of  tasks,  expenditures,  and  technological 
advances.  Unfortunately,  none  use  these  data  in  a  way  that  would 
ensure  the  proper  time  distribution  of  the  budget  allocation.  Most 
of  the  methods  concentrate  on  the  budget  requirements  for  the  next 
budget  period  without  regard  to  the  pattern  of  expenditures  required 
thereafter.  Those  methods,  such  as  TORQUE  and  FDL,  that  try  to  take 
account  of  the  future  pattern  of  budget  requirements,  do  so  by  means 
of  calculations  that  are  rather  arbitrary  and  without  firm  analytical 
bases . 

Without  a  demonstration  that  the  added  complexity  of  calculations 
such  as  the  timeliness  function  ensures  a  desirable  time  distribution 
of  the  budget,  the  complexity  is  probably  best  avoided.  Unless  the 
timing  aspects  of  tasks,  expenditures,  and  technological  advances  can 
be  incorporated  into  the  method  with  strong  and  verifiable  normative 
underpinnings,  simpler,  yet  visible,  listings  of  these  considerations 
would  be  more  desirable. 

8.  Risk 

a.  Discussion.  Risk  is  a  measure  of  the  extent  to  which  the 
various  aspects  of  the  actual  outcome  of  a  development  task  can  deviate 
from  the  results  predicted  for  the  task  before  its  execution. 
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In  some  respects,  a  development  task  is  much  like  a  production 
process.  A  group  of  inputs  is  assembled  and  organized  in  a  way  to 
generate  a  product,  in  this  -case  a  technological  advance.  However, 
if  any  one  characteristic  distinguishes  Exploratory  Development  from 
the  general  run  of  production  activities,  it  is  the  difference  in  the 
risk  that  confronts  the  development  manager.  The  development  process 
is  not  a  repetitive,  s ingle -product ,  manufacturing  process  in  which 
experience  with  the  pattern  of  inputs  and  products  eventually  cuts 
down  the  variance  between  predicted  and  actual  quantities.  Each 
development  task  is  a  new  experiment  with  significantly  different 
intended  products  and  little  comparative  experience  to  draw  upon  to 
specify  alternative  ’’production  processes.”  Consequently,  the  pos¬ 
sible  variance  between  predicted  and  actual  results  is  relatively 
much  greater  in  development. 

In  an  Exploratory  Development  task,  the  actual  resource  require¬ 
ments,  actual  timing  of  execution,  actual  elapsed  time  requirements, 
and  the  actual  technological  advance  achieved  can  vary  from  the 
predictions  made  of  these  characteristics  before  the  task  is  executed. 
These  are  jointly  distributed  characteristics,  each  of  which  can  be 
controlled  with  varying  degrees  of  precision  but  for  which  the  control 
of  any  one  is  achieved  at  some  loss  in  the  control  that  can  be  exer¬ 
cised  over  the  others.  The  variability  in  the  actual  outcomes  of 
these  characteristics  in  each  development  task  carries  through  the 
broader  development  process,  in  a  compounded  way,  into  the  ultimate 
configuration  of  the  weapon  systems  developed  and  their  fulfillment 
of  operational  requirements. 

The  risks  that  the  development  manager  faces  with  regard  to  each 
of  these  characteristics  and  the  relationships  among  them  should,  in 
some  way,  be  taken  into  account  in  the  allocation  method  if  it  is  to 
represent  at  all  adequately  the  real  allocation  problem.  When  the 
risks  associated  with  individual  characteristics  and  the  relation¬ 
ships  among  them  are  considered,  the  trade-offs  that  can  be  made  among 
the  characteristics  become  visible  so  that  they  can  be  evaluated  ex¬ 
plicitly  in  the  allocation  of  the  budget.  Consequently,  to  gain 
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assurance  of  a  satisfactory  technological  advance,  development  manage¬ 
ment  may  consciously  choose  to  finance  a  development  task  that  requires 
an  acceptable  time  for  completion  but  has  greater  expected  costs  and  a 
greater  possible  adverse  cost  variance  than  other  tasks  with  the  same 
objective.  Or  management  can  also  explicitly  consider  whether  the  as¬ 
surance  it  may  gain  in  achieving  a  particular  technological  advance  by 
increasing  the  funding  of  its  related  task  is  worth  the  assurance  that 
must  be  sacrificed  in  the  achievement  of  the  technological  advance 
sought  in  another  task  whose  funding  must  be  decreased. 

b.  Risk  in  the  Reviewed  Quantitative  Methods.  Most  of  the  meth¬ 
ods  that  were  reviewed  do  not  take  into  account  the  risk  involved  in 
the  many  aspects  of  the  development  process.  These  methods  set  out  the 
various  factors  as  though  the  results  of  any  projected  course  of  action 
are  known  precisely  beforehand.  Their  calculations  use  "point”  esti¬ 
mates  of  the  resources  employed  in. the  various  development  tasks,  the 
technological  advances  generated  by  the  tasks,  and  the  timing  of  the 
execution  of  the  tasks. 

The  FDL,  CAL,  Hercules,  and  toother  Service  methods  do  incor¬ 
porate  some  considerations  of  the  risks  that  are  involved,  but  the 
approaches  of  these  methods  are  quite  dissimilar. 

The  FDL  method  primarily  associates  risk  with  whether  the  tech¬ 
nological  advance  generated  by  a  task  will,  in  fact,  fit  into  the 
weapon  system  for  which  it  was  undertaken.  The  method  treats  the 
advance  generated  by  various  levels  of  effort  in  the  task  and  the 
timing  of  the  advance  as  if  they  are  known  precisely  but  each  known 
level  of  advance  as  if  it  fits  the  weapon  with  some  probability.  The 
task  and  advance  are  only  complete  once  the  latter  fits  the  weapon  for 
which  it  has  been  designed  with  a  probability  of  0.8.  At  best,  this 
treatment  only  alludes  to  the  risks  that  are  involved  in  Exploratory 
Development  tasks.  Having  chosen  the  0.8  probability  as  the  criterion 
of  success,  the  method,  in  effect,  has  converted  the  problem  to  one 
based  upon  "point"  estimates  of  the  factors. 
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The  CAL  method,  and  similarly  Another  Service  method,  incor¬ 
porates  risk  into  its  framework  in  terms  of  the  probability  of  suc¬ 
cess  of  the  individual  development  tasks  constituting  a  particular 
technical  approach  to  a  materiel  concept.  This  probability  of  success 
might  be  stated  as :  the  probability  that  an  advance  at  least  as  great 
as  the  advance  needed  for  the  technical  approach  will  result  from  the 
planned  expenditure  on  the  task  within  the  next  budget  period.  All 
the  tasks  within  a  given  technical  approach  must  succeed  in  these 
terms  for  the  technical  approach  to  be  successful,  and  at  least  one 
technical  approach  must  be  successful  for  the  completion  of  the  ma¬ 
teriel  concept.  Finally,  all  materiel  concepts  within  a  QMDO  must  be 
completed  successfully  for  the  fulfillment  of  the  QMDO.  By  this  se¬ 
quence  of  steps,  each  step  consisting  of  a  compounding  of  the  risks 
involved  in  the  more  basic  development  taks,  the  CAL  method  attempts 
to  evaluate  the  technical  risks  that  characterize  the  development  of 
a  complex  weapon  system. 

The  Hercules  method  attempts  to  carry  the  consideration  of  risk 
somewhat  further.  It  not  only  requires  estimates  of  the  technical 
risks  involved;  it  also  uses  estimates  of  the  marketing  and  profit¬ 
ability  risks.  Technical  risks  consist  of  engineering  estimates  of 
the  probabilities  that  new  research  must  be  performed  for  the  com¬ 
pletion  of  the  project  and  that  the  technical  goals  of  the  planned 
development  tasks  will  be  reached  within  the  proposed  time  and  funding 
patterns.  The  marketing  organization’s  input  to  the  method  consists 
of  estimates  of  the  probabilities  of  different  sales  volumes  and 
profit  margins  of  the  project’s  output  when  it  is  marketed. 

c.  Conclusions.  The  reviewed  quantitative  methods  largely 
ignore  the  risks  that  are  involved  in  the  real,  counterpart,  explora¬ 
tory  development  process.  The  only  application  of  a  fairly  thorough 
representation  of  the  risks  involved  in  development  is  the  Hercules 
method,  a  commercially  oriented  method  for  allocating  a  development 
budget.  The  CAL  method  does  have  a  good  framework  for  portraying  the 
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technical  risks  that  are  involved  at  different  levels  of  the  develop¬ 
ment  process.  However,  its  utility  for  allocating  a  military  Explora¬ 
tory  Development  budget  would  ultimately  depend  upon  the  quality  of 
the  information  used  in  its  application.  No  applications  of  the 
method  were  found,  so  final  judgment  must  be  postponed.  The  concept 
of  risk  introduced  into  the  FDL  method  barely  qualifies  as  a  serious 
consideration  of  the  matter.  Moreover,  the  limited  factors  to  which 
the  risk  concept  applies  do  not  give  the  manager  any  appreciation  of 
the  extent  to  which  the  actual  outcomes  of  the  development  efforts 
funded  can  deviate  from  those  outcomes  predicted  at  the  time  the  fund¬ 
ing  decisions  are  made. 

Risk  is  not  a  simple  factor  to  incorporate  into  quantitative 
methods  for  allocating  a  development  budget.  The  underlying  concepts 
are  difficult  to  formulate,  and  the  data  necessary  to  make  those  con¬ 
cepts  operational  are  difficult  to  obtain. 

For  the  most  part,  it  is  impossible  to  obtain  historical  data 
depicting  risk  that  apply  directly  to  the  actual  factors  involved  in  a 
current  budget  allocation  problem;  each  new  development  task  under¬ 
takes  to  discover  something  about  relatively  unknown  phenomena.  The 
data  that  are  used  are  generally  estimates  of  plausible  distributions 
of  the  outcomes  of  prospective  tasks.  Often  the  personnel  who  would 
execute  the  tasks  are  also  the  estimators  of  the  risks  involved. 

While  these  personnel  are  probably  the  best  informed  to  make  such 
estimates,  their  involvement  in  the  execution  of  the  task  does  not 
always  lend  to  their  making  estimates  as  objectively  as  possible. 

The  precise  mathematical  formulation  and  numerical  representation 
of  the  risk  involved  in  development  have  not  been  successfully  treated 
in  quantitative  methods  for  allocating  the  Defense  Exploratory  Devel¬ 
opment  budget.  Explicit  and  precise  incorporation  of  the  risk  factor 
into  such  methods  appears  to  require  more  development  work  itself. 
Obviously,  without  consideration  of  the  risk  involved,  a  method  cannot 
represent  for  management  the  range  of  trade-offs  that  is  important  to 
its  budget  allocation  decisions.  Until  more  precise  treatment  is 
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possible,  a  simpler  but  systematic  approach  would  be  desirable.  Such 
an  approach  might  consist  of  making  visible  some  evaluation  of  the 
chances  that  actual  outcomes  may  differ  from  predicted  outcomes  for 
a  few  basic  factors  in  the  development  process  without  attempting  to 
compound  these  risks  into  the  arithmetic  of  the  allocation. 

D .  CONSTRAINTS 
1.  Discussion 

In  a  quantitative  method  for  allocating  the  Exploratory  Develop¬ 
ment  budget,  constraints  are  the  limitations  that  the  analyst  and 
manager  apply  to  possible  courses  of  action  on  the  basis  of  consider¬ 
ations  outside  the  narrow  scope  of  Exploratory  Development.  They 
define  more  precisely  the  range  of  development  activities  that  can 
be  considered  seriously  by  setting  specific  limits  on  the  relation¬ 
ships  among  the  various  factors  in’ the  problem  or  by  setting  limits 
on  some  of  the  factors  directly. 

Obviously,  in  the  present  planning  context,  resolving  the 
allocation  of  resources  among  the  projects  of  the  Exploratory  Develop¬ 
ment  program  cannot  depend  upon  complete  systems  analyses  or  solution 
of  the  total  Defense  resource  allocation  problem.  Exploratory  Develop¬ 
ment  is  performed  much  too  early  in  the  evolution  of  a  new  system  for 
sufficient  information  to  be  available  for  a  complete  systems  analysis. 
Formulation  of  the  total  Defense  allocation  problem  in  a  rigorous 
planning  method  is  not  currently  practical,  if  at  all  desirable. 

To  be  at  all  manageable,  the  allocation  of  resources  within  Exploratory 
Development  must  be  treated  as  a  subproblem,  truncated  in  scope  but 
structured  consistently  with  the  more  general  Defense  allocation 
problem . 

The  analyst  can  use  constraints  to  truncate  the  Exploratory 
Development  problem  properly  and  to  set  it  up  in  such  a  way  that  it 
is  consistent  with  the  more  general  Defense  allocation  problem. 

One  type  of  constraint  consists  of  restrictions  on  the  progress 
that  might  be  sought  in  any  one  technology  or  on  the  configurations 
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of  the  weapon  systems  that  will  result  from  the  development  effort. 
Several  kinds  of  reasons  might  give  rise  to  such  bounds.  For  example, 
external  scientific  information  might  indicate  that  it  is  not  feasible 
to  achieve  some  level  of  a  critical  parameter  in  the  technology.  If 
the  information  indicates  that  conditions  change  quite  abruptly  at 
this  level,  a  restriction  might  be  used  to  prevent  expenditure  of 
further  effort  in  that  direction. 

Similar  constraints  might  be  placed  on  the  technical  configurations 
of  weapons  systems  based  on  systems  analyses  that  are  done  outside 
the  Exploratory  Development  planning  process.  Some  minimum  level  of 
a  performance  characteristic  in  a  weapon  system  might  be  imposed  on 
the  allocation  method  as  a  result  of  intelligence  estimates  of  enemy 
capabilities . 

Another  type  of  constraint  that  is  commonly  imposed  on  an  alloca¬ 
tion  method  is  a  limitation  on  the  resources  available.  This  may 
take  various  forms ,  including  a  restriction  on  the  overall  level  of 
expenditures  that  may  be  made  or  specific  restrictions  on  the  availa¬ 
bilities  of  individual  resources  such  as  facilities,  equipment, 
materials,  and  manpower.  Although  these  are  generally  in  terms  of 
upper  bounds,  the  condition  might  be  reversed  in  some  cases,  requiring 
that  a  minimum  amount  of  some  resource  be  used.  That  latter  situation 
can  commonly  arise  as  the  mix  of  Exploratory  Development  work  changes , 
giving  rise  to  a  changing  pattern  of  demand  for  resources.  However, 
other  considerations  may  lead  to  a  desire  to  stabilize  the  composition 
of  the  resources  on  hand  or  to  maintain  some  minimum  level  of  a  certain 
capability . 

Any  limitations  based  on  higher  policy-level  requirements  can  be 
imposed  on  the  method  to  limit  further  the  range  of  options  that  may 
be  chosen  for  the  Exploratory  Development  program.  However,  if  these 
are  properly  formulated  in  the  method,  the  analyst  should  be  able  to 
evaluate  the  costs  that  these  constraints  impose  on  the  program  in 
terms  of  higher  funding  requirements,  foregone  technological  progress, 
or  changed  weapon  design. 
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2 .  Constraints  in  the  Reviewed  Quantitative  Methods 

The  use  of  constraints  in  the  reviewed  methods  falls  into  three 
classes.  First,  some  of  the  methods  use  no  constraints  in  their  char¬ 
acterization  of  the  budget  allocation  problem.  Second,  some  of  the 
methods  use  only  one  constraint,  the  total  budget  that  is  available. 
Third,  a  few  of  the  methods  employ  a  number  of  constraints  on  a  vari¬ 
ety  of  aspects  within  the  allocation  problem. 

The  Industrial  Analog  and  Army  Missile  Plan  methods  fall  into  the 
first  type  (use  no  constraints)  but  they  are  significantly  different 
in  their  general  formulation.  Actually,  the  Army  Missile  Plan  is  less 
a  budget  allocation  method  than  it  is  an  information  system  for  making 
visible  the  derivation  of  the  technological  developments  that  are 
necessary  to  support  projected  weapon  systems.  Relationships  among 
the  various  factors  appear  to  keep  the  development  tasks  that  could  be 
undertaken  closely  linked  to  the  proposed  systems.  With  a  modest  pri¬ 
mary  objective  and  close  linkage  between  systems  and  tasks,  additional 
constraints  might  be  used  to  restrict  the  number  of  systems  supported 
for  any  one  operational  requirement.  However,  this  can  also  be  done 
successfully  by  limiting  the  number  of  proposed  weapon  systems. 

With  its  completely  misdirected  primary  objective,  the  Industrial 
Analog  would  need  a  set  of  constraints  that  would,  in  effect,  completely 
override  the  criteria  of  the  objective  to  ensure  an  allocation  of 
Exploratory  Development  funds  supporting  Defense  technology  needs. 

The  TORQUE,  NOL,  and  Hercules  methods  follow  the  second  approach, 
only  constraining  the  allocation  to  use  no  more  than  the  total  budget 
available.  The  Hercules  method  has  both  a  fairly  well-defined  primary 
objective  (to  maximize  company  profits)  and  a  relatively  precise  value 
measure  (profits).  These  might  be  expected  to  set  clearly  the  limits 
on  the  development  tasks  that  could  be  undertaken  except  possibly 
for  adequate  consideration  of  the  specific  skills  and  facilities 
available  to  the  company.  However,  in  an  application  of  the  method 
to  plan  one  year’s  program,  management  found  need  to  impose  an 
impromptu  constraint  on  the  allocation  generated  by  the  method.  In 
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that  particular  year  an  extremely  large  contract  appeared  probable 
in  the  very  field  of  rocket  propulsion  for  which  Hercules  was  well 
equipped.  If  the  potential  contract  were  incorporated  in  the  Hercules 
allocation  methodology  without  qualif ication,  its  immensity  would 
have  resulted  in  an  allocation  of  essentially  all  resources  toward 
the  one  contract.  Feeling  that  this  was  not  reasonable,  the  manage¬ 
ment  exercised  its  discretion  to  limit  the  allocation  to  that 
particular  project.  Such  a  constraint  undoubtedly  signifies  that 
some  other  factors  were  not  incorporated  into  the  method  satisfactorily. 

The  TORQUE  and  NOL  methods  also  only  take  into  account  the  over¬ 
all  limitation  on  the  budget  that  is  available  for  Exploratory 
Development  and  ignore  limitations  that  might  exist  on  the  availa¬ 
bility  of  specific  skills  and  facilities.  In  addition,  both  methods 
have  rather  nebulous,  unverifiable  primary  objectives  and  value 
measures  that  do  not  have  sufficient  inherent  properties  to  prevent 
the  method’s  generating  allocations  with  inconsistent  and  redundant 
tasks.  These  methods  make  no  provisions  for  constraints  that  would 
prevent  such  negative  features.  Systems  specifications  in  TORQUE 
characterize  this  problem. 

As  the  procedure  for  implementing  TORQUE  is  set  out  in  the  manuals 
and  was  followed  in  the  experiment,  the  TORQUE  method  does  not  give 
assurance  that  it  is  consistent  with  the  general  Defense  allocation 
problem.  The  principal  connection  of  the  Exploratory  Development 
planning  process  to  the  overall  Defense  allocation  problem  is  through 
the  projected  weapon  systems  that  Exploratory  Development  is  expected 
to  support  in  its  own  program.  Consequently,  TORQUE’S  consistency 
with  the  larger  problem  depends  upon  the  choice  of  the  weapon  systems 
used  as  the  basis  for  determining  the  technology  of  objectives  of  Ex¬ 
ploratory  Developments.  TORQUE  basically  has  little  or  nothing  to  do 
with  the  procedure  for  choosing  the  set  of  prospective  weapon  systems 
that  should  guide  the  planning  of  technology  development.  But  it  also 
does  not  clearly  incorporate  those  prospective  major  weapon  systems 
that  should  be  determined  by  the  systems  analysts,  into  the  stream  of 
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that  should  be  determined  by  the  systems  analysts,  into  the  stream  of 
data  and  assumptions  used  in  its  calculations .* 

Besides  limiting  the  total  allocation  to  the  budget  available  for 
Exploratory  Development  work,  the  FDL  method  does  take  into  account 
the  availability  of  some  specific  resources  for  this  work.  In 
published  reports,  the  method  considered  restrictions  on  the  amounts 
of  in-house  and  contract  engineering  manpower  that  could  be  employed 
in  the  resulting  tasks.  This  is  an  important  consideration  and,  in 
principle,  establishes  that  some  specialized  resources  might  limit 
the  size  or  rate  of  expansion  of  the  program.  However,  the  FDL 
method  has  the  same  shortcoming  as  TORQUE  and  NOL  regarding  the 
specification  of  the  systems  that  should  be  considered  in  the 
derivation  of  the  development  tasks . 

In  addition  to  a  constraint  on  the  total  budget  that  can  be 
allocated,  the  CAL  method  employs  QMDO  and  resource  constraint 
concepts.  The  QMDO  constraint  ensures  that  at  least  one  technical 
approach  to  each  materiel  concept  of  a  specific  QMDO  will  be  included 
in  the  final  program  if  a  QMDO  constraint  is  stated  for  that  particular 
QMDO.  Unfortunately,  the  probability  of  success  of  the  QMDO  is  not 
at  all  considered  in  this  regard.  Consequently,  a  very  low-cost,  low 
probability-of -success  configuration  of  technical  approaches  might  be 
retained  in  the  program  for  a  "necessary”  QMDO. 

The  principal  purpose  of  the  resource  constraint  is  to  make  sure 
that  the  funding  of  a  particular  field  establishment  does  not  fall 
below  some  predetermined  level.  The  particular  formulation  of  the 
resource  constraints  can  lead  to  perverse  results.  A  technical  approach 
will  not  be  eliminated  from  the  program  if  its  elimination  will 
violate  a  resource  constraint;  that  is,  if  its  elimination  would  lead 


Although  the  particular  matter  falls  outside  the  scope  of  this  study, 
recognizing  that  the  determination  of  the  weapon  systems  options  is 
so  important  to  technology  development  planning  raises  the  further 
question  of  whether  the  current  state  of  systems  planning  is  suffi¬ 
cient  to  fulfill  this  function. 


to  less  being  expended  at  a  field  establishment  than  the  minimum 
previously  determined.  This  is,  at  best,  a  rather  crude  "make-work" 
provision  that  can  only  lead  to  higher  development  costs  than  neces¬ 
sary  for  prospective  weapon  systems.  It  can  actually  prevent  distri¬ 
bution  of  the  work  among  the  field  establishments  on  the  basis  of 
their  absolute  or  comparative  efficiencies  in  the  technical  areas. 
Consequently,  the  resource  constraints  prevent  the  field  establish¬ 
ments,  as  a  whole,  from  achieving  the  maximum  possible  "research  and 
development  output"  under  the  given  funding  conditions. 

Minimum  levels  of  effort  at  the  field  establishments  are  set  to 
satisfy  objectives  other  than  the  development  of  the  specific  QMDOs 
being  investigated  at  any  one  time.  Both  sets  of  objectives  could 
be  better  served  if  they  were  not  confounded  through  this  particular 
formulation  of  the  resource  constraints . 

3 .  Conclusions 

Constraints  are  the  means  by  which  the  method  formulator  can 
introduce  considerations  into  the  analysis  of  the  allocation  problem 
to  make  it  correspond  more  closely  to  the  real  counterpart  budgeting 
problem.  They  can  also  be  used  to  fix  limits  on  the  range  of  factors 
that  must  be  taken  into  account  in  the  allocation  problem.  In  this  way, 
constraints  truncate  the  scope  of  the  problem  to  fit  more  closely  the 
primary  concerns  of  Exploratory  Development  by  imposing  on  it  outside 
policy  and  technical  decisions. 

The  formulators  of  the  methods  reviewed  did  not  exploit  this 
analytical  feature  as  effectively  as  might  be  desired.  In  most  cases, 
it  seems  that  the  formulators  were  conservative  in  their  recourse  to 
these  devices.  Of  course,  the  formulators  pay  a  price  in  using  con¬ 
straints.  First,  constraints  are  often  difficult  to  translate  into 
an  acceptable  mathematical  form  and  their  numerical  content  is  often 
difficult  to  estimate.  Second,  constraints  generally  add  to  the 
computational  burden  of  solving  the  desired  allocation  of  the  budget. 

However,  even  in  the  CAL  method,  where  constraints  are  used  fairly 
freely,  the  particular  forms  of  the  QMDO  and  resource  constraints  do  not 
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effect  very  well  the  external  decisions  dealing  with  QMDO  completion 
and  facilities  funding. 

E.  DECISION  ALGORITHM 

1.  Discuss  ion 

In  the  final  step  of  a  quantitative  method,  all  the  factors  con¬ 
sidered  in  the  problem  are  combined  with  the  objectives  and  constraints 
in  a  decision  algorithm  to  determine  the  funding  of  each  proposed  de¬ 
velopment  effort.  This  decision  algorithm  is  basically  a  formal  set 
of  rules  for  calculating  the  level  of  funding.  It  uses  the  mathemati¬ 
cal  representations  of  the  factors  in  a  way  that  reflects  the  consider¬ 
ations  management  should  make  in  forming  its  choices. 

Algorithms  used  in  allocation  methods  in  general  can  be  quite  di¬ 
verse  in  their  mathematical  complexity  and  rigor.  Very  simple  rules 
of  thumb  applied  consistently  qualify  as  algorithms  as  much  as  the 
very  complex  mathematical  techniques  involved  in  combinatorial  search 
routines,  Lagrangian  multiplier  methods,  and  linear,  nonlinear,  and 
dynamic  programming. 

For  the  most  part,  the  criteria  of  the  usefulness  of  an  algorithm 
consists  of  whether  it  treats  the  factors  of  the  problem  consistently, 
whether  it  combines  the  factors  and  represents  their  relationships  in 
a  way  that  reflects  the  real  counterpart  to  the  allocation  problem, 
and  whether  the  computational  procedure  is  consistent  with  the  primary 
objective  of  the  allocation  method.  In  a  formal  mathematical  algorithm, 
the  last  of  these  criteria  is  generally  demonstrated  through  a  proof 
that  the  mathematical  procedure  converges  to  an  allocation  that  best 
satisfies  the  primary  objective.  The  first  two  criteria  are  satisfied 
if  the  mathematical  expressions  for  the  factors  and  their  relationships 
are  formulated  properly  and  cover  the  proper  range  of  considerations. 

2 .  Decision  Algorithms  in  the  Reviewed  Quantitative  Methods 

Even  within  the  relatively  small  number  of  quantitative  methods 
reviewed  for  this  investigation,  a  wide  range  of  decision  algorithms 
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is  used.  This  range  includes  a  simple  listing  of  development  tasks, 
linear  programming,  and  combinatorial  search  routines. 

The  Army  Missile  Plan  uses  a  simple  listing  of  the  proposed  de¬ 
velopment  tasks.  The  ordering  of  the  tasks  in  the  list  depends  upon 
(1)  the  initial  operational  capability  date  of  the  weapon  system  in 
which  the  results  of  the  task  would  be  used,  (2)  the  ordinal  importance 
of  the  weapon  system  revealed  in  Army  planning  documents,  and  (3)  the 
indispensibility  of  the  task  results  to  the  weapon  system.  Once  the 
order  of  the  tasks  is  established,  the  funds  are  allocated  to  each 
task  in  descending  order  on  the  list  according  to  the  resource  require¬ 
ments  of  each,  until  the  total  budget  is  exhausted. 

The  simplicity  of  this  procedure  is  quite  appealing.  The  task 
order  and  the  steps  in  the  allocation  are  visible  and  readily  compre¬ 
hensible.  The  obvious  drawback  is  that  it  does  not  represent  very 
well  the  full  range  of  options  that  the  management  of  Exploratory 
Development  must  take  into  consideration.  Alternative  development 
tasks  with  the  same  technological  objective  cannot  be  evaluated  readily. 
Also  the  procedure  does  not  take  into  account  how  some  higher  priority 
tasks  might  be  stretched  out  at  lower  funding  rates  in  order  to  permit 
the  starting  of  tasks  that  have  somewhat  lower  ordinal  priorities. 

Little  objection  can  be  raised  with  the  mathematical  procedures 
of  the  algorithms  used  by  the  NOL,  FDL,  and  Hercules  methods.  The  NOL 
method  uses  a  Lagrangian  multiplier  type  of  algorithm,  the  FDL  a  linear 
programming  algorithm,  and  the  Hercules  method  a  combinatorial  search 
routine.  In  the  NOL  and  FDL  methods,  as  long  as  the  mathematical  for¬ 
mulations  of  the  factors  and  the  relationships  among  the  factors  have 
the  proper  forms,  the  algorithms  lead  to  best  solutions.  Consequently, 
any  objections  to  them  must  stem  from  how  well  these  mathematical  for¬ 
mulations  represent  the  real  counterpart  conditions  surrounding  the 
factors.  Most  of  the  shortcomings  in  the  treatment  of  the  various 
factors  in  these  methods  have  already  been  reviewed  above.  However, 
some  observations  on  these  methods  deserve  emphasis  in  the  present 
context. 
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The  Index  of  Advance,  Fig.  3  above,  plays  a  crucial  role  in  the 
NOL  method;  the  particular  form  of  the  curve  assumed  in  the  descrip¬ 
tion  is  necessary  if  the  algorithm  is  to  apply  properly.  However,  the 
field-  survey  of  this  investigation  indicated  that  other  shapes  of  such 
value  functions  may  describe  better  the  real  counterpart  technological 
progress  (cf.  Fig.  9).  If  these  other  shapes  are  more  appropriate, 
the  NOL  algorithm  can  no  longer  ensure  that  its  allocation  satisfies 
best  the  primary  objective  specified  for  the  method. 

In  the  FDL  method,  the  weapon  systems  and  technical  goals  devised 
are  extremely  important  components  of  the  functions  that  define  the 
military  value  of  each  development  task  (Section  III-D-5).  Inspection 
of  the  form  of  this  function  indicates  that  the  value  of  a  task  is  de¬ 
pendent  upon  the  number  of  weapon  systems  and  technical  goals  it  sup¬ 
ports,  as  it  should  be.  However,  without  some  additional  constraints, 
a  well-chosen,  and  arbitrary,  proliferation  of  the  right  weapon  systems 
and  technical  goals  can  weight  the  recommended  allocation  incorrectly 
in  favor  of  some  tasks. 

As  is  the  case  with  any  algorithm,  the  final  test  of  the  combina¬ 
torial  search  used  in  the  Hercules  method  is  the  resulting  allocation. 
In  this  regard,  it  might  be  reemphasized  that  management  rejected  the 
allocation  generated  in  one  application  of  the  method  when  its  solution 
indicated  that  practically  all  of  the  funds  should  be  devoted  to  one 
very  large  project.  Management  wanted  a  somewhat  wider  distribution 
of  effort.  Without  a  much  deeper  investigation  of  the  method,  such  a 
conflict  might  be  attributed  to  either  a  faulty  allocation  algorithm 
or • incomplete  consideration  of  some  factors  in  the  problem. 

The  algorithm  in  the  CAL  method  begins  with  a  program  that  in¬ 
cludes  all  the  technical  approaches  to  all  the  QMDOs .  It  then  elimi¬ 
nates  technical  approaches,  one  at  a  time,  according  to  the  decrease 
in  the  "expected"  value  of  the  program  per  dollar  cost  that  would  re¬ 
sult  from  removing  the  technical  approach  from  the  program.  Those 
approaches  that  would  result  in  the  smallest  loss  of  "expected"  value 
per  dollar  cost  are  eliminated  first.  The  elimination  proceeds  in 
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order  as  long  as  a  resource  constraint  or  QMDO  constraint  is  not  vio¬ 
lated.  If  elimination  of  a  technical  approach  would  violate  one  of 
these  constraints,  that  technical  approach  is  retained  in  the  program, 
and  the  next  in  the  order  is  tested.  The  procedure  continues  until 
the  total  cost  of  the  remaining  technical  approaches  falls  within  the 
limit  of  the  total  budget  that  is  available. 

The  simplicity  of  such  an  elimination  procedure  is  quite  attrac¬ 
tive.  Unfortunately,  it  gives  absolutely  no  assurance  that  the  final 
program  recommended  satisfies  the  primary  objective  at  least  as  well 
as  other  possible  combinations  of  technical  approaches.  In  other 
words,  some  of  the  eliminated  approaches  might  be  substituted  into  the 
program  in  a  way  that  does  not  require  more  funds  than  the  available 
budget  and  that  results  in  greater  expected  value.  The  designers  of 
the  method  admit  that  the  algorithm  may  not  generate  optimum  solutions 
(Ref.  16,  p.  B-8)  but  did  not  investigate  the  problem  further  because 
of  the  short  term  of  their  study  contract.  Such  a  shortcoming  obviously 
places  doubt  on  the  advisability  of  applying  the  algorithm  of  the 
method. 


The  decision  algorithm  used  in  TORQUE  funds  each  task  in  incre¬ 
ments  on  the  basis  of  the  increase  in  utility  per  dollar  increase  in 
cost  that  results  from  funding  the  next  increment  of  each  task.  The 
task  increment  with  the  highest  ratio  of  added  utility  to  added  cost 
is  funded  first. 

The  TORQUE  algorithm  has  shortcomings  that  arise  .both  in  the  way 
that  the  various  factors  are  combined  in  the  utility  function  and  in 
the  particuler  steps  followed  to  determine  the  task  increments  that 
should  be  funded. 


The  most  fundamental  components  of  the  utility  function  relate 
the  development  of  a  single  technology  objective  to  a  single  weapon 
system.  These  are  shaped  quite  regularly,  as  is  shown  in  Fig.  12(a). 
In  this  type  of  function,  the  funding  and  the  timeliness  function 
value,  t.,  ,  vary  within  narrowly  defined  limits.  The  timeliness  func- 

3  k  P 

tion  is  the  principal  determinant  of  the  shape  of  u  first 
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year  funding,  F.^,  is  varied,  the  late  and  early  points  on  the  timeli¬ 
ness  function  (Fig.  2,  above)  generate  the  two  corresponding  corners 
of  Fir  The  segments  of  F  to  the  left  of  late  and  to  the  right  of 
early  are  the  results  of  the  timeliness  function  segments  after  the 
latest  need  date  and  before  the  earliest  need  date.  The  segment  be¬ 
tween  the  late  and  early  points  falls  on  a  ray  from  the  origin  having 
a  slope  S  F  where  V\h  is  the  operational  requirement  weight 

assigned  to  the  particular  weapon,  is  the  criticality  of  tech¬ 

nology  objective  1  to  this  weapon  system,  and  £  Fyl  is  the  total  amount 
that  still  must  be  spent  to  achieve  technology  objective  1  in  subse¬ 
quent  years,  y. 


(a)  ONE  TECHNOLOGY,  ONE  SYSTEM 


(b)  ONE  TECHNOLOGY,  MANY  SYSTEMS 


FIRST  YR.  FUNDING  TO  TECHNOLOGY  OBJECTIVE  k  =  1 
FOR  SYSTEM  j  =  1 


FIRST  YR.  FUNDING  TO  TECHNOLOGY  OBJECTIVE  k  =  1 
FOR  SYSTEM  j 


FIGURE  12.  TORQUE  Utility  Functions 

Questions  about  the  appropriateness  of  the  timeliness  function 
were  raised  above  in  Section  IV-C-7-b.  Specifically,  for  present  pur¬ 
poses,  the  segment  for  the  period  before  the  early  date  indicates  that 
a  technology  objective  developed  more  than  two  years  before  that  date 
has  absolutely  no  value.  This  implies  that  what  might  be  learned  in 
the  development  of  this  technology  objective  has  absolutely  no  impact 
for  what  might  be  developed  in  other  technology  objectives  related  to 


150 


the  same  system.  It  also  Implies  that  in  the  derivation  of  the  u_.^ 
functions  for  all  weapon  systems  j,  the  planner  knows  with  certainty 
that  those  systems  exhaust  the  set  of  weapons  to  which  the  technology 
objective  1  will  be  applied. 


The  segment  of  the  timeliness  function  for  the  period  after  the 
late  date  implies  that  completion  dates  can  be  known  fairly  precisely. 
However,  one  of  the  most  important  elements  of  risk  in  the  development 
process  is  the  time  that  will  be  required  to  bring  an  effort  to  suc¬ 
cessful  completion.  Time  estimates  in  Exploratory  Development  are 
typically  a  few  years;  when  combined  with  Advanced  Development,  these 
times  may  reach  four  to  eight  years.  If  the  time  estimates  are  in 
error  by  two  years  or  more,  the  precision  of  the  timeliness  function 
downgrades  the  value  of  late  developments  inaccurately . 

Figure  12(b)  depicts  the  ideal  aggregate  utility  function  for  a 
single  technology  objective,  combining  individual  functions  for  a 
large  number  of  weapon  systems  with  overlapping  need  dates  for  this 
particular  technology  objective.  However,  the  aggregation  cannot  en¬ 
sure  the  regular  shape  shown  in  Fig. 12(b).  A  single  technology 
objective  supporting  two  weapon  systems  that  have  somewhat  separated 
initial  operating  capability  dates  could  have  a  utility  function 
shaped  like  the  one  in  Fig.  13. 

Inspection  of  the  utility  function  (Section  III-B-5)  indicates 
quite  clearly  that  the  value  of  any  technology  objective  can  be  in¬ 
creased  simply  by  including  in  the  analysis  a  larger  number  of  weapon 
systems  using  it.  That  the  weapon  systems  may  or  may  not  be  effective 
in  their  missions,  that  all  the  weapon  systems  may  apply  only  to  a 
siingle  operation  or  requirement,  or  that  the  weapon  systems  chosen  in 
the  program  might  fulfill  the  requirement  several  times  over  are  not 
important  to  the  utility  function.  In  fact,  it  is  quite  unable  to 
discern  such  irregularities. 

These  characteristics,  in  turn,  indicate  very  strongly  a  basic, 
and  serious,  specification  error  in  the  utility  function.  It  is  an 
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error  that  can  possibly  be  corrected,  but  not  by  a  few,  simple  modifi¬ 
cations  . 


FIGURE  13.  Utility  Function  of  a  Technology  Objective  Supporting 
Two  Weapon  Systems 


The  number  of  weapon  systems  actually  considered  in  the  formal 
allocation  routine  of  TORQUE  depends  upon  the  discretion  of  the  I.D. 
team.  If  it  so  desired,  the  team  could  introduce  several  variations 
on  a  particular  weapon  system,  changing  only  one  or  a  few  component 
technology  objectives,  calling  each  a  separate  system,  and  thereby 
(consciously  or  not)  biasing  the  allocation  of  resources  towards 
those  technology  objectives  held  constant  across  the  variations. 

The  strong  independence  or  additivity  requirement  of  the 
Churchman-Ackof f  approximate  measure  of  value  also  carries  over  into 
the  algorithm.  As  a  consequence  of  it,  the  value  that  will  generally 
be  attributed  to  the  joint  occurrence  of  two  technology  objectives 
used  in  separate  but  supporting  weapon  systems  would  be  understated. 
This  understatement  will  result  if  the  analyst  who  devises  the  weights 
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treats  the  weapons  in  their  separate  roles,  as  he  should  under  the 
instructions  for  the  consistent  application  of  the  value  measure,  and 
does  not  try  to  anticipate  their  supporting  roles.  Such  an  understate¬ 
ment  will  make  these  technology  objectives  less  attractive  for  incre¬ 
ments  of  funding  than  probably  would  be  consistent  with  an  efficient 
development  program. 


Sensitivity  analyses,  which  show  that  small  changes  in  the  program 
solution  result  from  rather  wide  variations  in  the  weights,  only  beg 
this  problem.  As  far  as  can  be  discerned,  the  sensitivity  analyses  re¬ 
tained  the  relative  rankings  that  were  given  the  separate  operational 
requirements  in  the  Churchman-Ackof f  measurement  procedure,  whereas  com¬ 


plementarity  between  weapon  systems  essentially  breaks  down  the  rankings. 

In  the  computations  followed  to  determine  the  task  increments 

that  should  be  funded,  TORQUE  does  not  use  all  of  the  points  on  the  u^ 

function.  Instead,  it  calculates  selected  points  along  that  function 

for  consideration  in  the  computations.  To  do  this,  the  procedure  first 

determines  the  maximum  first  year  funding,  F^,  that  will  be  "allowed” 

for  each  technology  objective.  is  the  particular  F^.^,  the  first 

year  fundings  of  the  alternate  budgets  proposed  by  pthe  Technology 

u£ 

Team  (cf .  Fig.  10),  that  gives  the  maximum  value  — £ —  .  In  Fig.  13, 

E 1  j  k  F 

the  above  procedure  is  equivalent  to  finding  the  value  of  u^,  corre¬ 
sponding  to  an  F,  .,  ,  that  is  connected  to  the  origin  by  the  ray  of 

IJK 

greatest  slope.  In  this  case  the  Fj_jk  satisfying  this  condition  is 
Fjj^, .  For  the  allocation  iterations,  the  procedure  then  calculates 
and  records  the  values  of  u^  at  increments  of  O.lnF^,  where  n  =  1,2, 

...,  10. 

As  is  evident  from  Fig.  13,  the  routine  by  which  F  ^  is  chosen 
can  exclude  from  the  eligible  options  funding  levels  that  could  con¬ 
tribute  substantial  utilities  to  the  exploratory  development  program. 

In  that  example,  the  higher  funding  levels,  by  which  the  technology 
objective  is  developed  more  quickly  for  use  in  high  utility  systems 
with-  sooner  IOC  dates,  would  be  excluded  from  consideration  once  Flk 
is  set  at  F.,,  . 
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The  necessity  of  choosing  a  specific  value,  F^,  is  quite  doubt¬ 
ful  in  the  first  place,  but  the  particular  procedure  and  criterion 
used  lead  to  highly  undesirable  results. 

The  computation  procedure  is  beset  by  another  pitfall.  As  was 

.  p 

indicated  above,  for  the  allocation  iterations,  the  values  of  uv  are 
calculated  and  recorded  only  at  the  discrete  intervals  O.lnF^  for 
n  =  1,2,  ...,  10.  Once  an  initial  level  of  funding  is  allocated  to 
a  technology  objective,  the  addition  to  that  level  of  a  further  incre¬ 
ment  of  O.lF-j^  is  evaluated  on  the  basis  of  the  utility  that  would  be 
added  per  dollar  of  the  funding  increment.  If  the  additional  funds 

p 

resulted  in  a  funding  level  that  corresponds  to  a  "trough”  in  the  u, 

K. 

function,  that  is,  an  actual  decline  in  the  utility,  that  increment, 
and  any  further  increments,  in  the  funding  of  the  technology  objective 
would  be  ignored.  In  Fig.  14,  the  initial  funding  level  of  the  tech- 

np 

nology  objective  is  O.lnF^,  corresponding  to  "tangency"  with  u^ 

(cf.  the  Appendix).  If  the  funding  increment,  O.IF^,  were  to 
result  in  a  funding  level  of  A(=0.1nF^  +  O.IF^),  the  corresponding 
utility  increment  would  be  negative  and  additional  funding  rejected, 
despite  the  higher  utilities  possible  at  much  higher  funding  levels. 

3 .  Conclusions 

This  investigation  has  focused  principally  on  the  substantive 
issues  involved  in  the  formulation  of  a  quantitative  method  for  allo¬ 
cating  the  Exploratory  Development  budget.  Therefore,  it  has  placed 
somewhat  less  emphasis  on  the  formal  algorithms  that  are  used  to  com¬ 
pute  a  recommended  allocation.  That  emphasis  does  not  mean,  however, 
that  the  algorithm  employed  in  a  particular  method  is  not  important  or 
that  all  algorithms  are  equally  useful. 

Obviously,  the  formulation  of  the  planning  model  and  the  algorithm 
that  will  be  employed  in  its  computation  are  somewhat  interdependent. 
However,  the  position  of  this  investigation  is  that  without  an  appro¬ 
priate  conceptual  framework  of  the  allocation  problem,  speculation 
about  an  algorithm  remains  largely  a  matter  of  mathematics. 
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FIGURE  14.  TORQUE  Algorithm 


Rather  elaborate  mathematical  procedures  are  used  in  most  of  the 
methods  that  were  reviewed.  Some  of  these  are  new  applications  of 
existing  optimization  techniques,  others  are  mathematical  methods  that 
give  no  assurance  they  converge  to  the  best  allocations,  and  still 
others  are  extensive  computer  search  routines.  Both  the  CAL  method 
and  TORQUE  algorithms  have  shortcomings  that  cast  serious  doubt  on 
their  abilities  to  arrive  at  the  best  possible  allocations. 

Despite  the  increased  availability  of  mathematical  programming 
algorithms,  devising  an  appropriate  technique  may  remain  a  serious 
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problem  in  the  implementation  of  a  quantitative  allocation  method. 
Consequently,  a  more  thorough  study  of  this  feature  of  a  quantitative 
allocation  method  should  be  made  in  conjunction  with  further  in-depth 
study  of  the  exploratory  development  planning  problem. 

F.  FINDINGS  ON  EACH  METHOD 

This  section  contains  comments  on  each  method  that  do  not  fit 
into  the  general  framework  of  analysis  used  above.  Where  appropriate, 
these  are  combined  with  the  conclusions  reached  in  the  preceding  anal¬ 
ysis  of  the  common  features  of  the  methods  to  support  conclusions 
about  the  usefulness  of  each  method. 

1.  Industrial  Analog 

The  Industrial  Analog  is  a  highly  simplistic  method.  Assuming 
that  data  are  readily  available  for  each  of  the  variables,  including 
commercial  and  Defense  product  complexities  and  lifetimes,  and  that 
relatively  stable  and  precise  functional  relationships  among  these 
variables  can  be  estimated  statistically,  little  subsequent  informa¬ 
tion  and  calculation  would  appear  to  be  necessary  to  determine  the 
allocation  of  6.1  and  6.2  funds.  It  also  requires  little  involvement 
by  higher  management  in  the  distribution  of  funds  among  projects, 
tasks,  or  work  units  within  the  general  technical  fields. 

Moreover,  the  Industrial  Analog  has  obvious  intuitive  appeal. 
Because  Defense  Research  and  Exploratory  Development  problems  are  not 
tractable  and  generally  cannot  be  evaluated  easily,  some  kind  of 
emulation  of  the  apparent  success  by  American  industry  would  be  highly 
attractive . 

Several  reasons  were  given  above  why  this  emulation  and  the  data 
used  to  carry  it  out  could  be  expected  to  lead  to  antiproductive  re¬ 
sults  for  military  Research  and  Exploratory  Development.  However, 
one  more  aspect  of  the  data  deserves  mention.  The  ratio  of  company- 
funded  R&D  expenditures  to  net  sales  used  in  the  method  is  an 
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industry-wide  average,  computed  from  individual  company  data  that 
vary  quite  widely.  Although  they  are  averages  themselves,  the  data 
in  Table  7  give  some  indication  of  this  variation  by  company  size. 

Given  this  variability,  a  strong  rationale  would  be  needed  to  support 
use  of  the  industry  average  in  a  scheme  for  determining  DoD  R&D  funding. 
Because  of  the  particular  input  requirements,  progress  in  a  technical 
field  might  well  have  been  produced  by  companies  spending  more  than 
the  industry  average.  On  the  other  hand,  companies  spending  the 
average  or  more  in  another  technical  field  may  be  using  their  R&D 
resources  inefficiently  so  that  similar  "progress"  could  be  achieved 
with  lower  expenditures. 

The  prescriptions  of  the  Industrial  Analog  method  could  only 
contribute  to  the  efficient  achievement  of  national  security  goals 
by  accident.  Considering  the  nature  of  the  method’s  objectives,  the 
factors  taken  into  account,  and  the  relationships  among  the  factors 
that  make  up  its  basic  analytical  framework,  several  severe  and  highly 
unlikely  conditions  must  be  satisfied  before  its  allocations  would 
even  be  fortuitously  consistent  with  general  Defense  objectives.  The 
same  considerations  suggest  that,  inasmuch  as  commercial  objectives 
may  be  inappropriate  and  the  ratios  of  company- funded  R&D  expenditures 
to  industry  sales  are  only  remotely  normative  measures  of  behavior, 
Industrial  Analog  guidance  may  be  absolutely  wrong. 

2.  TORQUE  Method 

The  procedures  recommended  in  the  TORQUE  manuals  and  the  composition 
of  the  Interdisciplinary  Team  impose  on  the  overall  planning  process 
the  condition  that  the  relationships  between  weapon  systems  and  tech¬ 
nologies  be  considered  systematically.  This  is  very  important  for 
developing  the  priorities  to  direct  the  work  that  should  be  done  in 
Exploratory  Development  to  support  future  weapon  systems. 

Also  the  organizations  and  functions  of  the  Interdisciplinary 
and  Technology  Teams  require  close  coordination  among  the  systems 
analysts  and  technologists.  This  interaction  should  promote  a  sharper 
awareness  of  each  other’s  work  on  the  part  of  both.  The  systems 

157 


TABLE  7.  COMPANY  FUNDS  FOR  R&D  PERFORMANCE  AS  PERCENT  OF  NET  SALES 
IN  R&D -PERFORMING  MANUFACTURING  COMPANIES, 

BY  INDUSTRY  AND  SIZE  OF  COMPANY,  1965 


Companies  with  total  employment  of- 

INDUSTRY 

TOTAL 

Less  than 
1,000 

1,000  to 
4,999 

8,000  or 
more 

Total. 


Food  and  kindred  products.., 

Textiles  and  apparel . 

Lumber,  wood  products,  and 

furniture . . 

Paper  and  allied  products.. 
Chemicals  &  allied  products, 


Industrial  chemicals. 
Drugs  &  medicines.... 
Other  chemicals 


Petroleum  refining  &  extraction 

Rubber  products . 

Stone,  clay,  &  glass  products.. 
Primary  metals . 

Primary  ferrous  products.. 
Nonferrous  and  other 

metal  products . 


0.4 

0.4 

0.5 

0.5 

0.5 

0.3 

0.9 

0.6 

4.2 

3.7 

4.1 

4.0 

6.7 

5*6 

2.6 

1.7 

Fabricated  metal  products, 

Machinery . 

Electrical  equipment  and 
communications . . 


Communication  equipment  & 
electronic  components. . . 
Other  electrical  equipment 


Motor  vehicles  and  other 
transportation  equipment.. 

Aircraft  and  missiles . 

Professional  and  scientific 
instruments . . 


1.2 

1.1 

1.0 

1.4 

3.2 

1.6 

1.6 

4.2 

3.5 

3.0 

2.2 

3.8 

4.2 

4.7 

1.0 

4.5 

3.0 

2.2 

2.3 

3.2 

Scientific  &  mechanical 
measuring  instruments.. 

Optical,  surgical,  photo¬ 
graphic,  &  other 
instruments . 


jother  manufacturing  industries. 


(a)  Not  separately  available  but  included  in  tota 
Source:  Ref.  20. 


158 


analysts  should  bring  the  current  and  projected  states  of  technology 
more  directly  to  bear  upon  the  designs  of  prospective  systems.  The 
technologists  should  be  able  to  appreciate  better  the  functions  that 
specific  technologies  can  serve  in  weapon  system  development  and 
direct  their  efforts  accordingly. 

While  TORQUE  has  these  commendable  procedural  features,  it  omits 
some  other  important  considerations.  It  makes  no  provision  for  taking 
into  account  the  non-Defense  effort  being  expended  upon  the  tech¬ 
nologies  that  would  support  projected  weapon  systems.  Systematic 
consideration  of  such  work  would  prevent  undesirable  parallel  efforts 
and  waste.  Also  TORQUE  makes  no  provision  to  prevent  wide  fluctua¬ 
tions  in  the  funding  (and  consequently  in  the  employment  of  specialized 
personnel  and  facilities)  or  particular  technological  areas  from  year 
to  year.  Large,  rapid  adjustments  of  this  sort  may  be  highly  undesir¬ 
able,  particularly  if  maintaining  some  level  of  in-house  capability 
were  necessary  to  support  minimum  liaison  with  the  scientific  community, 
procurement  specification  operations,  and  contract  monitoring. 

Changes  in  the  method  are  probably  possible  that  would  meet  these 
objections.  However,  no  simple  modifications  could  be  made  that  would 
remedy  the  shortcomings  reviewed  above  regarding  (a)  the  assignment  of 
weights  to  the  operational  requirements,  (b)  the  treatment  of  the  tech¬ 
nological  composition  of  the  weapon  systems  in  a  way  that  gives  rise 
to  both  redundant  and  incomplete  technological  developments  for 
supporting  prospective  weapons,  (c)  the  ignoring  of  the  risk  of  failure 
in  each  task,  (d)  the  treatment  of  the  timing  of  technological  de¬ 
velopments,  and  (e)  the  rules  of  choice  used  in  the  decision  algorithm. 

All  of  these  shortcomings  are  sufficiently  fundamental  that  the  analytical 
framework  fails  to  address  the  budget  allocation  problem  properly. 

3.  Naval  Ordnance  Laboratory  Method 

Striking  parallels  exist  between  the  NOL  method  and  TORQUE.  The 
military  worths  of  the  EDGs  in  the  NOL  method  are  the  same  as  the 
weights  assigned  to  the  operational  requirements  in  TORQUE.  The 
utility  of  a  technology  to  an  EDG  in  NOL  is  similar  to  the  criticality 
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of  a  technology  to  a  weapon  system  in  TORQUE.  Also,  both  TORQUE  and 
the  NOL  method  omit  any  consideration  of  the  risks  of  failure  in  any 
exploratory  development  task. 

In  addition,  the  NOL  method  does  not  lend  itself  to  time-phasing 
the  budgets  that  should  be  allocated  to  the  various  technologies.  Nor 
does  the  way  it  takes  into  account  the  relationship  of  a  technology 
to  an  EDG  ensure  that  the  expenditure  derived  by  the  algorithm  for  any 
technology  will  produce  the  level  of  the  pacing  parameter  of  that  tech¬ 
nology  required  in  the  EDG. 

Finally,  while  the  mathematics  of  the  decision  algorithm  are 
sound  when  applied  to  the  functions  that  are  assumed  in  the  NOL  method 
to  depict  technological  advance,  evidence  (such  as  that  behind  Fig.  9) 
indicates  that  the  shape  assumed  for  these  functions  may  be  inappro¬ 
priate.  If  the  shape  assumed  in  the  NOL  method  is  inappropriate,  the 
algorithm  does  not  retain  the  property  that  it  will  generate  the  best 
possible  allocation. 

4.  Air  Force  Flight  Dynamics  Laboratory  Method 

The  FDL  method  also  treats  many  of  the  factors  it  takes  into  account 
in  ways  that  resemble  the  approach  of  TORQUE.  Its  primary  objective 
and  control  variable  are  virtually  the  same;  the  procedure  it  uses  to 
assign  weights  to  operational  requirements  is  also  based  on  the  Church- 
man-Ackoff  approximate  measure  of  value;  its  breakdown  of  systems  into 
component  technologies  parallels  TORQUE;  it  uses  exactly  the  same  weights 
to  measure  the  criticality  of  a  technical  objective  to  a  system;  and 
it  employs  the  same  timeliness  function  to  evaluate  the  time -phasing 
of  the  development  tasks. 

On  the  other  hand,  the  FDL  method  tries  to  take  into  account  varia¬ 
tions  in  the  performance  of  the  different  operational  requirements  by 
the  different  weapon  systems.  It  tries  to  introduce  some  consideration 
of  the  risk  that  a  particular  technology  objective  may  fit  the  design 
of  projected  systems.  Also,  the  FDL  method  employs  a  long-standing, 
tried  and  proven  decision  algorithm:  linear  programming. 
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In  sum,  however,  the  FDL  method  does  not  address  the  problem  of 
allocating  the  Exploratory  Development  budget  more  satisfactorily 
than  TORQUE.  The  faults  in  their  common  features  are  so  basic  to  an 
adequate  formulation  of  the  problem  that  the  FDL  method  gives  no  more 
assurance  a  program  it  generates  will  be  more  consistent  with  or  pro¬ 
mote  better  national  defense.  Its  treatment  of  risk  is  perfunctory, 
giving  management  little  idea  of  how  actual  outcomes  of  development 
efforts  could  deviate  from  expected  outcomes.  Although  it  makes 
allowance  for  the  performance  of  individual  weapon  systems  in  differ¬ 
ent  operational  requirements,  it  does  not  ensure  any  type  of  balance 
in  the  technical  support  given  the  range  of  the  operational  require¬ 
ments.  In  other  words,  the  resulting  development  effort  could  well 
be  directed  at  fulfilling  only  one  or  a  few  operational  requirements. 

5.  Cornell  Aeronautical  Laboratory  Method 

The  CAL  method  is  a  departure  from  the  general  approaches  taken 
in  the  TORQUE,  NOL,  and  FDL  methods.  It  links  rather  strongly  the 
different  technological  efforts  to  the  future  weapon  systems  they 
support.  The  other  methods  impute  value  to  the  technological  efforts 
on  the  basis  of  the  weapons  they  support  and  subsequently  drop  that 
linkage.  CAL*s  treatment  of  risk  is  noteworthy  and  probably  merits 
further  development.  It  takes  into  account  the  probability  of  success 
of  each  development  task  and  compounds  those  for  each  system  to  derive 
the  probability  that  the  system  will  be  successful.  The  method  identi 
fies  the  different  resources  used  in  the  development  tasks,  maintains 
a  record  of  those  resource  patterns,  and  ultimately  ensures  the  use 
of  more  than  some  floor  level  or  of  less  than  some  ceiling  level  of 
each  resource  if  management  wishes. 

The  CAL  method  does  have  faults,  however,  that  are  sufficient  to 
prevent  its  acceptance  as  a  satisfactory  approach  to  the  budget  alloca 
tion  problem.  It  does  rely,  in  its  current  formulation,  upon  the 
assignment  of  values  to  the  various  QMDOs  but  gives  no  instructions 
on  how  this  assignment  should  be  made.  Although  this  shortcoming 
might  be  overcome  by  judicious  use  of  QMDO  constraints,  requiring  the 
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development  of  certain  QMDOs,  such  a  remedy  must  be  worked  out  in  some 
detail  before  it  could  be  applied  confidently.  Beyond  the  valuation 
problem,  technologies  and  tasks  are  treated  too  rigidly.  In  its 
present  formulation,  the  CAL  method  treats  technical  approaches  to 
a  particular  materiel  concept  as  possible  substitutes  for  each  other. 
All  materiel  concepts  to  a  QJiDO  are  necessary  to  fulfill  the  QMDO. 

Much  additional  development  of  the  method  would  be  necessary  to  expand 
the  formulation  so  that  in  its  solution  (a)  trade-offs  might  be  possi¬ 
ble  among  technical  approaches,  (b)  sequential  and  parallel  develop¬ 
ment  strategies  could  be  derived,  and  (c)  substitutions  and  synergisms 
among  materiel  concepts  could  be  expressed  and  utilized. 

Finally,  another  decision  algorithm  must  be  applied  to  its  formu¬ 
lation  of  the  factors  that  bear  upon  the  allocation  problem.  The 
heuristic  procedure  recommended  in  its  current  stage  of  development 
could  generate  solutions  that  are  inferior  to  other  possible  solutions. 
The  steps  in  the  algorithm  do  not  guarantee  that  they  are  not  ignoring 
combinations  of  tasks  or  QMDOs  that  would  better  serve  the  objectives 
and  criteria  of  the  method. 

6 .  Hercules  Method 

The  Hercules  method  was  the  only  method  among  those  reviewed 
that  was  not  specifically  formulated  to  deal  with  the  problem  of 
allocating  the  budget  for  Defense  Exploratory  Development.  It  was 
designed,  however,  to  analyze  the  problem  of  allocating  the  budget 
for  development  work  of  a  similar  nature  with  commercial  application. 
Being  in  a  commercial  setting,  the  formulators  of  the  Hercules  method 
could  readily  use  measures  of  expected  sales  and  profits  as  the  ulti¬ 
mate  measures  of  value  for  the  development  work  involved.  These  are 
fairly  easy  to  define,  estimate,  and  verify  among  the  management. 
Moreover,  they  can  be  transformed  into  an  operational  primary  objective 
the  maximization  of  the  profits  that  the  development  effort  can  ulti¬ 
mately  generate  for  the  company.  The  Department  of  Defense  does  not 
have  similar  measures  of  value  readily  available  but  more  intensive 
study  of  the  application  of  such  a  method  could  help  to  clarify  how 
such  a  concept  as  cost  savings  might  be  used  in  a  parallel  fashion. 
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Other  aspects  of  the  Hercules  method  also  probably  warrant  fur¬ 
ther  study  for  possible  applicability  to  allocating  the  Defense  Ex¬ 
ploratory  Development  budget.  For  example,  the  description  of  this 
method  indicated  that  the  formulators  attempt  to  take  into  account 
both  the  trade-offs  that  might  be  made  among  technological  components 
of  prospective  systems  and  the  synergistic  effects  these  components 
may  have  on  one  another.  The  timing  of  the  development  effort  is  in¬ 
vestigated  in  terms  of  management Ts  general  time  preferences  rather 
than  by  some  rigid  functional  rule.  Also,  the  method  introduces  risk 
at  several  levels,  such  as  prospective  sales,  margins,  technical  suc¬ 
cess,  and  timing. 

Consideration  of  risk  may  be  one  of  the  weaker  features  of  the 
Hercules  method.  Use  of  expected  values  or  some  particular  level  of 
probability  to  characterize  the  outcome  of  development  events  may 
hide  very  interesting  and  important  information,  e.g.  the  chances 
of  incurring  losses  of  unacceptable  sizes.  Management Ts  overriding 
the  allocation  determined  by  the  method  to  avoid  concentrating  the 
total  development  effort  on  one  project  may  indicate  that  this  weak¬ 
ness  has  been  recognized. 

7 .  Army  Missile  Plan 

Along  with  the  two  following  methods,  the  Army  Missile  Plan  is 
a  planning  method  that  has  a  much  more  modest  objective  than  the 
methods  discussed  above.  Those  methods  attempt  in  some  way  to  "opti¬ 
mize”  the  Defense  Exploratory  Development  program.  The  Army  Missile 
Plan  attempts  only  to  set  out  explicitly  the  technological  develop¬ 
ment  efforts  that  the  laboratory  personnel  judge  to  be  necessary  to 
support  the  engineering  and  production  of  weapons  proposed  for  future 
deployment.  The  efforts  that  are  ultimately  funded  are  those  con¬ 
tributing  in  important  ways  to  systems  having  the  highest  priorities. 

Taking  this  restricted  approach  has  two  important  results.  First, 
it  limits  the  number  of  options  from  which  management  can  choose  the 
development  program  to  be  implemented.  This  is  obviously  a  shortcoming 
because,  as  is  the  case  in  almost  any  decision  situation,  the  chance  of 
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increasing  the  contribution  of  the  Exploratory  Development  program  to 
national  defense  objectives  should  be  higher  if  the  decision  maker  can 
consider  a  broader  set  of  options.  This  chance  should  be  higher  unless 
the  options  are  expanded  to  the  point  that  they  are  so  numerous  and 
irrelevant  that  they  create  confusion  and  some  chance  of  choosing 

counterproductive  programs. 

/ 

Second,  the  limited  approach  avoids  entanglement  in  a  number 
of  difficult  problems.  Complex  value  measures  need  not  be  devised, 
the  trade-offs  and  synergisms  among  technologies  are  not  considered, 
complicated  but  possible  time-phasings  of  tasks  are  ignored,  numeri¬ 
cal  descriptions  of  the  risks  involved  are  not  needed,  and  intricately 
structured  decision  algorithms  are  superfluous.  While  all  of  these 
features  are  important  aspects  of  any  method  for  allocating  the  Explor¬ 
atory  Development  budget,  their  exclusion  may  be  less  misleading  than 
their  being  incorporated  with  highly  inappropriate  treatment. 

Although  the  Army  Missile  Plan  does  not  aim  to  devise  the  "optimum 
Exploratory  Development  program,  the  information  that  it  makes  avail¬ 
able  to  program  management  is  an  obvious  benefit. 

8 .  Air  Force  Directorate  of  Laboratories  Plan 

The  DOL  plan  is  even  more  modest  than  the  Army  Missile  Plan.  It 
has  a  similar  objective  but  it  does  not  resort  to  any  scheme  of 
priorities  among  the  operational  requirements  or  the  systems  that 
would  be  used  in  those  requirements.  Further,  the  DOL  plan  does  not 
try  to  assign  some  measure  of  the  contribution  of  any  technology  ob¬ 
jective  to  the  weapon  systems.  Technical  planning  objectives  are  set 
out  apparently  without  analyses  of  possible  trade-offs  among  the  system 
components.  The  method  does  not  even  propose  to  rank  the  program  ele¬ 
ments  or  projects  to  set  up  some  order  of  funding.  Essentially,  the 
DOL  plan  is  a  bare-boned  trace  of  technological  needs  without  any 
reference  to  the  priorities  of  those  needs  or  the  risks  that  might 
characterize  the  tasks  undertaken  to  fill  them. 


9 .  Army  Research  Plan 


Like  the  Army  Missile  Plan,  the  Army  Research  Plan  is  only  partly 
quantitative  in  its  analysis  of  the  allocation  of  the  Exploratory  De¬ 
velopment  budget.  However,  the  two  methods  differ  fairly  widely  in 
some  important  aspects.  Both  methods  utilize  a  scheme  of  priorities 
to  characterize  the  relative  importance  of  individual  operational  re¬ 
quirements.  They  both  also  assign  weights  to  the  different  tech¬ 
nological  efforts  to  indicate  the  contributions  of  the  technological 
efforts  to  operational  requirements  or  weapon  systems. 

The  Army  Missile  Plan  explicitly  refers  to  specific  weapons 
proposed  for  deployment  to  derive  its  pattern  of  technological  effort 
while  the  Army  Research  Plan  skips  over  systems  and  relates  technologies 
directly  to  operational  requirements.  The  Missile  Plan  attempts  to 
trace  specifically  the  funding  of  individual  tasks  whereas  the  Research 
Plan  directs  its  attention  to  a  somewhat  broader  question.  It  is  con¬ 
cerned  with  the  apparent  general  adequacy  of  funding  for  the  broader 
technological  effort  encompassed  by  a  program  element  which  is  an 
aggregate  of  tasks  and  projects.  A  third  major  difference  between 
the  Missile  Plan  and  the  Research  Plan  is  their  approach  to  the  funding 
decision.  The  Missile  Plan  ranks  the  individual  tasks  for  specific 
funding  on  the  basis  of  their  priorities  and  technical  contributions, 
whereas  the  Research  Plan  has  a  number  of  staff-generated  qualitative 
analyses  that  serve  as  the  basis  for  recommendations  on  whether  the 
program  element  should  be  given  more  emphasis.  The  staff  does  not 
rank  in  any  order  the  program  elements  that  are  recommended  for 
greater  emphasis  to  suggest  priorities  even  within  that  group. 

The  Research  Plan  is  basically  a  method  for  structuring  informa¬ 
tion  and  conveying  it  to  management.  The  few  aspects  of  the  alloca¬ 
tion  problem  that  it  does  quantify  are  not  used  in  computations  to 
make  precise  recommendations  on  the  distribution  of  funds.  Conse¬ 
quently,  the  information  is  carried  along  quite  visibly  in  the  de¬ 
liberations.  Such  a  method  can  be  an  attractive  approach  to  the  al¬ 
location  problem  while  more  quantitative  methods  are  being  developed. 
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10.  Another  Service  Method 


This  Other  Service  method  is  quantitative,  having  some  features 
similar  to  those  in  the  TORQUE,  NOL,  FDL,  and  CAL  methods.  Like  TORQUE, 
NOL,  and  FDL,  it  uses  a  set  of  operational  requirements  in  its  deriva¬ 
tion  of  the  technological  advances  that  the  Exploratory  Development 
effort  should  try  to  achieve.  Like  those  same  methods,  it  also  re¬ 
quires  that  a  measure  of  value  be  assigned  the  various  operational 
requirements.  The  same  problems  that  characterize  the  measures  of 
value  used  in  the  more  quantitative  methods  above  apply  equally  to 
this  method.  The  procedure  for  devising  these  weights  assumes  that 
the  operational  requirements  are  all  independent  of  each  other  and 
that  they  are  not  fulfilled  regardless  of  the  amount  of  development 
work  performed  to  support  them. 

Like  the  NOL  method,  this  method  does  not  make  reference  to 
prospective  weapons  to  derive  the  Exploratory  Development  program. 
Technologies  are  applied  directly  to  the  operational  requirements  in 
the  method  without  an  evaluation  of  how  the  technologies  make  up  specific 
weapons.  Consequently,  the  technologies  are  considered  as  though  they 
are  quite  independent  of  each  other  and  are  assigned  weights  that  the 
analysts  judge  to  reflect  the  isolated  contribution  of  the  technology 
to  the  operational  requirement.  As  in  the  TORQUE,  NOL,  and  FDL  methods, 
such  treatment  leaves  little  room  for  considerations  of  how  the  tech¬ 
nologies  interact  either  as  substitutes  or  as  reinforcements  for  each 
other . 

This  method  resembles  the  CAL  method  to  some  extent  in  the  way 
that  it  treats  the  probabilities  of  the  tasks  achieving  their  tech¬ 
nical  objectives.  The  probabilities  of  success  of  all  tasks  directed 
at  a  specific  technical  objective  are  compounded  to  estimate  the  prob¬ 
ability  of  at  least  one  succeeding.  However,  unlike  the  procedure  at 
CAL,  these  probabilities  of  success  for  the  technical  objectives  are 
not  used  in  any  way  to  estimate  the  chances  of  progress  in  the  dif¬ 
ferent  operational  requirements. 
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This  feature  emphasizes  the  independence  attributed  to  the  vari¬ 
ous  technical  objectives.  The  technical  objectives  derive  their  values 
from  the  operational  requirements  they  support,  but  no  relationships 
among  the  technologies  are  necessary  to  fulfill  that  support. 

The  joint  consequences  of  the  independence  and  insatiability 
attributed  to  each  operational  requirement  and  the  independence  of 
the  development  tasks  are  that  the  Exploratory  Development  program 
may  be  directed  at  technologies  that  support  the  single  operational 
requirement  with  the  highest  weight  but  that  do  not  fit  together  in 
any  feasible  weapon  design.  In  other  words,  without  some  specially 
designed  constraints  to  prevent  such  an  outcome,  this  method  is  quite 
open  to  generating  a  development  program  that  is  unbalanced  and  un¬ 
coordinated.  For  example,  all  Exploratory  Development  work  might  be 
directed  toward  the  development  of  technologies  that  contribute  only 
to  strategic  defense,  but  the  results  in  propulsion,  guidance,  and 
warhead  design  would  be  incompatible  in  any  specific  weapon. 

G.  GENERAL  DISCUSSION 

This  section  summarizes  the  conclusions  and  recommendations  that 
were  reached  in  the  above  analysis,  as  they  apply  to  the  more  quanti¬ 
tative  methods,  to  the  less  quantitative  methods,  and  generally  to 
both  groups  of  methods . 

1.  More  Quantitative  Methods 

The  more  quantitative  methods  (Industrial  Analog,  TORQUE,  Naval 
Ordnance  Laboratory,  Flight  Dynamics  Laboratory,  Cornell  Aeronautical 
Laboratory,  Hercules,  and  Another  Service)  attempt  to  structure  the 
allocation  problem  and  express  its  factors  almost  exclusively  in  math¬ 
ematical  terms.  They  calculate  a  single,  most  desirable  distribution 
of  the  development  budget.  All  of  these  methods,  except  for  Hercules, 
were  devised  to  address  the  problem  of  allocating  the  budget  of  the 
Defense  development  program. 

These  methods  are  relatively  complex;  they  treat  factors  in  a 
precise  fashion.  They  link  the  derivation  of  the  development  program 
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to  weapon  systems  that  have  been  proposed  for  deployment  against  future 
operational  requirements.  Consequently,  these  methods  provide  a  frame¬ 
work  for  systematic  liaison  among  systems  specialists,  technologists, 
and  research  managers.  This  should  help  to  broaden  the  development 
options  that  are  considered  in  the  derivation  of  the  final  program. 

Complexity  has  the  drawback,  however,  that  the  specific  treatment 
given  the  various  factors  in  a  method  may  be  obscure  to  everyone  ex¬ 
cept  the  few  specialists  who  formulate  the  method. 

Despite  their  complexity  and  their  attention  to  detail  and  pre¬ 
cision,  these  methods  generally  treat  too  many  of  the  important  factors 
in  the  allocation  problem  inadequately  or  inaccurately.  None  of  the 
formulators  of  the  more  quantitative  methods  meant  for  application  to 
the  allocation  of  Defense  development  resources  appear  to  have  been 
aware  of  the  importance  of  delineating  properly  the  scope  and  primary 
objective  of  their  methods.  They  have  all  used  a  very  broad-gauged 
objective  for  the  allocation  of  the  development  resources  (to  devise 
the  development  program  with  the  maximum  military  utility).  Such  an 
objective  should  open  up  the  options  that  might  be  considered  for  the 
program.  However,  the  linkages  in  the  models  among  the  development 
efforts,  prospective  weapon  systems,  operational  requirements,  and  ob¬ 
jectives  are  specified  in  ways  that  the  choices  made  in  the  develop¬ 
ment  program  can  also  settle  more  comprehensive  Defense  program  issues 
than  should  properly  be  determined  by  the  development  program.  The 
choice  is  made  among  the  alternative  development  efforts  in  such  a  way 
that  it  can  also  determine  the  emphasis  that  will  be  placed  on  the  de¬ 
velopment  of  specific  weapon  systems  and  consequently  on  the  fulfill¬ 
ment  of  the  various  operational  requirements.  This  order  of  determina¬ 
tion  is  the  reverse  of  good  decentralized  programming. 

None  of  these  methods  use  constraints  on  the  set  of  development 
options  that  is  considered  in  the  allocation  problem  in  such  a  way 
that  prevents  this  reversal. 

Most  of  these  methods  treat  individual  development  efforts  as 
though  they  are  independent  of  each  other,  particularly  in  the  sense 
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that  they  are  not  interrelated  as  components  of  proposed  weapons.  A 
similar  independence  is  attributed  to  other  factors  as  well.  Weapon 
systems  are  treated  as  though  they  have  little  interaction  with  each 
other  and  operational  requirements  are  treated  as  though  they  are  ful¬ 
filled  in  isolation  from  each  other. 

Independence  among  the  development  efforts,  weapons,  and  opera¬ 
tional  requirements  is  necessary  to  the  procedure  that  is  used  to  im¬ 
pute  value  to  these  factors.  To  the  extent  that  the  independence  is 
not  real,  the  values  imputed  to  the  factors  are  inaccurate  and  mis¬ 
leading. 

None  of  the  more  quantitative  methods  specify  with  precision  the 
development  costs  that  should  be  analyzed  in  the  allocation  problem. 
They  are  particularly  remiss  in  setting  the  ground  rules  for  what  de¬ 
velopment  resources  should  be  taken  into  account  and  for  the  treatment 
of  expenditures  that  must  be  made  in  the  future.  How  should  real  es¬ 
tate  that  is  Government -owned  and  used  in  a  development  effort  be 
treated?  Should  the  costs  of  military  manpower  employed  in  develop¬ 
ment  be  treated  as  a  development  cost?  How  should  appropriations 
needed  in  future  budget  years  be  treated? 

None  of  the  more  quantitative  methods  address  the  important  mana¬ 
gerial  problem  of  how  the  actual  technical,  cost,  and  timing  outcomes 
of  the  development  effort  may  jointly  deviate  from  the  technical, 
cost,  and  timing  estimates  made  at  the  time  the  allocation  is  decided. 
They  all  employ  point  estimates  of  these  factors  without  providing  for 
the  collection  of  data  on  them  as  efforts  progress  or  for  the  updating 
of  the  estimates  in  subsequent  allocations. 

The  more  quantitative  methods  for  allocating  resources  within 
Defense  Exploratory  Development  are  at  a  rudimentary  stage  in  their 
own  development.  Each  of  the  methods  reviewed  has  identified  the  im¬ 
portant  considerations  in  the  allocation  problem.  However,  as  was 
pointed  out  immediately  above,  the  particular  manner  in  which  indi¬ 
vidual  factors  have  been  expressed  mathematically  and  incorporated 
into  the  specific  analytical  frameworks  is  only  a  first,  and  often 
inaccurate,  effort. 
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Consequently,  none  of  the  more  quantitative  methods  reviewed 
should  be  applied,  in  their  present  formulations,  to  allocating  re¬ 
sources  within  Defense  Exploratory  Development. 

The  inapplicability  of  these  particular  methods,  however,  should 
not  discourage  further  study  and  development  of  such  methods.  Properly 
formulated  methods  could  provide  valuable  assistance  to  management  for 
its  determination  of  the  development  program. 

Further  study  should  be  made  of  methods  that  have  been  devised 
for  application  to  resource  allocation  within  industrial  research.  Of 
the  more  quantitative  methods,  the  Hercules  method  structured  the 
problem  and  treated  the  important  factors  most  appropriately. 

Specifying  an  appropriate  primary  objective  is  a  particularly 
difficult  obstacle  to  the  formulation  of  a  method  for  allocating  re¬ 
sources  within  Defense  Exploratory  Development.  The  Defense  problem 
has  no  readily  adaptable  counterpart  to  the  profit-maximization  ob¬ 
jective  used  in  the  Hercules  method.  However,  other  industrial  methods 
could  be  studied  for  techniques  that  may  be  used  to  express  objectives 
applied  to  the  limited  scope  of  a  company1 s  development  program.  In 
large  companies  these  objectives  may  be  derived  and  decentralized  from 
that  of  the  overall  company  objectives  in  a  way  that  resembles  the 
subordinate  role  of  Defense  Exploratory  Development. 

Further  study  should  also  be  made  of  the  adaptation  of  a  cost- 
minimization  objective  to  the  allocation  of  resources  within  Defense 
Exploratory  Development.  This  investigation  should  begin  with  the 
studies  of  cost  savings  imputed  to  the  C-141  transport  and  the  AN/SPS-48 
radar  that  were  reported  in  HINDSIGHT  (Ref.  4).  The  objective  assigned 
to  Exploratory  Development  would  be  to  generate  the  technological 
progress  supporting  the  development  of  weapon  systems  that  would  pro¬ 
duce  the  greatest  cost  savings  while  fulfilling  a  given  set  of  mis¬ 
sions  . 
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2 .  Less  Quantitative  Methods 

The  less  quantitative  methods  (Army  Missile  Plan,  Air  Force  Di¬ 
rectorate  of  Laboratories  Plan,  and  Army  Research  Plan)  do  not  try  to 
express  mathematically  some  important  factors  in  the  allocation  prob¬ 
lem.  They  place  much  greater  reliance  upon  managerial  judgment  to  take 
into  account  such  factors.  These  methods  may  recommend  an  allocation 
of  resources  among  proposed  development  efforts  but  without  the  numer¬ 
ical  precision  of  the  more  quantitative  methods  and  without  contending 
that  the  recommendation  is  the  single,  most  desirable  allocation. 

The  less  quantitative  methods  attempt  primarily  (1)  to  record  ex¬ 
plicitly  information  on  some  relevant  considerations  that  must  be  made 
in  determining  the  allocation  of  the  development  budget  and  (2)  to 
transmit  this  information  visibly  to  all  the  levels  of  management  in¬ 
volved  in  determining  that  allocation. 

These  methods  have  more  narrowly  circumscribed  objectives  than 
the  more  quantitative  methods.  They  focus  tightly  on  deriving  those 
technical  advances  that  would  support  a  very  limited  and  specific  set 
of  future  weapon  systems.  As  a  result,  the  allocations  chosen  using 
one  of  these  methods  should  be  internally  consistent  and  provide  an 
integrated  set  of  technologies  in  support  of  some  set  of  weapon  sys¬ 
tems  proposed  for  future  deployment. 

On  the  other  hand,  being  tightly  focused,  the  less  quantitative 
methods  are  not  likely  to  take  into  consideration  a  set  of  possible 
and  useful  development  efforts  that  is  as  extensive  as  the  set  made 
visible  in  the  more  quantitative  methods. 

These  methods  leave  to  management's  judgment  the  consideration 
of  some  factors  that  are  recognized  to  be  important  but  for  which  no 
precise  numerical  expression  is  devised.  Management  must  elect  whether 
and  how  to  take  into  account  such  factors  as  the  contribution  of  a 
particular  weapon  system  to  an  operational  requirement,  the  importance 
of  a  technical  advance  to  a  weapon,  the  relevance  of  various  kinds  of 
costs,  and  the  possibility  that  actual  cost,  timing,  and  technical  re¬ 
sults  may  deviate  from  those  predicted  at  the  time  the  allocation  decision 
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is  made.  Undoubtedly,  these  factors  contain  very  substantial  subjec¬ 
tive  components.  However,  if  the  consideration  they  receive  is  not 
carefully  specified  and  they  are  not  made  entirely  explicit,  much  of 
the  decision  process  for  allocating  the  development  resources  is  left 
obscure.  As  a  result,  the  rationale  for  the  chosen  allocation  remains 
largely  implicit  and  difficult  to  review  with  the  ultimate  consequence 
of  appearing  arbitrary. 

Because  of  the  limited  extent  of  the  set  of  options  they  take 
into  consideration  and  the  lack  of  precision  in  their  treatment  of 
some  important  factors  in  the  allocation  problem,  these  methods  may 
not  choose  the  development  programs  best  supporting  higher  order  De¬ 
fense  goals. 

Less  quantitative  methods  such  as  those  reviewed  in  this  study 
are  fairly  well  developed.  More  or  less  formally,  this  type  of  analy¬ 
sis  of  the  allocation  problem  has  provided  the  background  for  actual 
resource  distribution.  Consequently,  it  has  benefitted  from  the  ex¬ 
perience  of  its  practitioners  and  the  informal  feedback  that  the  lat¬ 
ter  have  received  in  the  budgetary  process. 

Undoubtedly,  many  research  managers  should  find  some  one  of  these 
methods  a  convenient  framework  for  organizing  the  information  that 
they  want  readily  available  when  they  must  decide  their  allocation  of 
development  resources. 

Further  development  of  the  less  quantitative  methods  would  prob¬ 
ably  give  them  a  greater  resemblance  to  the  more  quantitative  methods. 
The  less  quantitative  methods  must  describe  more  explicitly  and  pre¬ 
cisely  such  important  factors  in  the  allocation  problem  as  the  tech¬ 
nical,  cost,  and  timing  risks  involved  in  the  development  of  tech¬ 
nology.  Specific  expressions,  possibly  numeric,  for  these  factors 
and  the  relationships  of  technologies  to  weapon  system  characteristics 
would  permit  the  decision  maker  to  consider  them  more  systemat ically 
in  choosing  his  allocation  of  resources. 
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3 .  More  Quantitative  and  Less  Quantitative  Methods 

All  of  the  methods  that  were  reviewed  approach  the  development 
programming  problem  in  a  prospective  sense  only.  That  is,  they  are 
concerned  with  planning,  before  the  fact,  the  distribution  of  develop¬ 
ment  resources  that  should  be  made  in  the  future  to  proposed  develop¬ 
ment  efforts . 

However,  by  itself,  prospective  planning  may  not  very  effectively 
influence  the  course  of  actual  development  effort.  Planning  must  be 
integrated  with  techniques  for  controlling  actual  resource  use.  One 
of  the  principal  aids  to  control  is  a  record  of  resource  consumption 
and  outcomes  that  follows  the  same  framework  and  terms  used  for  the 
prospective  plans.  None  of  the  reviewed  methods  provide  for  a  sys¬ 
tematic  recording  of  the  actual  cost,  time,  and  technical  outcomes  of 
the  funded  development  efforts  in  a  format  that  would  facilitate  com¬ 
parison  with  the  estimates  underlying  the  planned  allocation  of  re¬ 
sources.  This  could  be  an  invaluable  source  of  information  for  sub¬ 
sequent  applications  of  one  of  the  methods  'and  for  making  adjustments 
to  the  allocations  that  were  made  in  the  past  to  current  development 
efforts . 

A  similar  kind  of  default  on  the  part  of  the  methods'  formulators 
is  the  basis  for  the  concern  expressed  above  about  the  objectivity  or 
reliability  of  the  measures  of  value  (Section  IV-C-l-b)  and  the  meas¬ 
ures  of  the  relationships  of  technologies  to  weapon  systems  (Section 
IV-C-4-b)  that  are  used  in  the  methods.  For  factors  such  as  these, 
the  formulators  have  not  established  measurement  procedures  that  can 
be  used  with  confidence  that  a  given  set  of  observers  making  the 
measurements  would  arrive  at  the  same  quantities  for  the  same  factors 
under  fixed  conditions  but  at  different  times.  Nor  do  the  procedures 
provide  for  measuring  the  variation  that  might  arise  in  the  measure¬ 
ment  of  these  factors  by  different  sets  of  observers. 

Similarly,  but  on  a  larger  scale,  the  formulators  have  not  pro¬ 
posed  that  the  methods  be  subject  to  a  serious  test  of  their  overall 
reliability.  Such  a  test  would  involve  measuring  the  extent  to  which 
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a  given  set  of  decision  makers  might  arrive  at  different  resource  al¬ 
locations  (1)  applying  at  different  times  a  given  method  to  the  same 
external  threat,  development  opportunities,  and  other  conditions,  or 
(2)  applying  different  methods  to  a  fixed  context  at  the  same  time. 
Other  measures  of  reliability  could  be  made  from  having  different  sets 
of  decision  makers  apply  a  given  method,  then  different  methods,  to  a 
fixed  set  of  conditions. 

One  further  problem  with  the  methods  arises  out  of  these  measure¬ 
ment  difficulties.  Because  the  reliability  of  the  measures  of  some 
important  factors  is  not  demonstrated  and  because  the  objectives 
adopted  in  some  of  the  more  quantitative  methods  are  quite  hazy,  there 
is  no  way  to  verify  retrospectively  whether  the  chosen  program  actually 
achieved  its  objective.  How  can  the  utility  of  the  technological  ad¬ 
vances  achieved  by  a  program  be  measured  in  a  way  that  can  be  compared 
with  the  utilities  estimated  at  the  time  the  allocation  is  made? 

The  same  type  of  difficulty  extends  to  measuring  the  relative 
benefits  of  using  one  method  as  opposed  to  any  other.  The  formulators 
of  the  methods  have  not  indicated  how  a  test  might  be  devised  to  demon¬ 
strate  that  the  program  resulting  from  the  allocation  reached  by  a 
particular  method  is  in  any  way  superior  to  the  program  generated  by 
another  method,  or  without  any  method. 
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APPENDIX 


TORQUE  BUDGET  ALLOCATION  ALGORITHM 

The  TORQUE  budget  allocation  algorithm  consists  of  two  major 
operations ,  each  of  which  is  further  divided  into  a  set  of  procedural 
steps.  The  first  major  operation  is  the  explicit  expression  of  each 
technology  objective  ,T utility  function.’’  The  second  major  operation 
uses  these  ’’utility  functions”  in  an  iterative  procedure  for  distribut¬ 
ing  the  available  funds  among  the  technology  objectives. 


Technology  Objective  Utility  Functions 

The  first  major  operation  of  the  allocation  algorithm  is  the 
explicit  generation  of  the  utility  function  for  each  technology 
objective  through  the  following  steps. 


For  each  technology  objective 


(1) 


(2) 


(3) 


Calculate  the  values  of  u,  ,  using  in  the  technology 
objective  ’’utility  function”  the  first  year  fundings,  P^jk’ 
from  the  alternate  budgets  proposed  by  the  Technology  Team 

to  support  the  different  weapon  systems, 

r  F  1  F 

among  those  calculated  in  (1)  and  designate 


Find  Max 

j 


°k 

*1* 


the  corresponding  P^jk  aS  FlkJ 

F 

Calculate  the  values  of  at  discrete  funding  levels,  using 
in  the  technology  objective  utility  function  increments  of 
one-tenth  the  value  of  F.^>  or  O.IF-^j 


0.1nFlk  =  cumulation  of  n  increments  of  F^,  where  n  =  1,2, 

...,10, 
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np 

=  value  of  the  technology  objective  utility  function 

at  O.lnF-.  , 
lk  ’ 

p 

(4)  Record  the  values  of  u^  calculated  in  (3)  as  the  technology 
objective  utility  function. 


Budget  Allocation  Iterations 

The  iterative  procedure  for  distributing  the  program  budget  is 
carried  out  in  the  following  steps. 

(1)  From  the  utility  function  of  each  technology  objective 
recorded  in  the  last  step  above,  calculate 


(6) 


nF 

“k 


which  u^  is  the  maximum  at  O.lnF^,  set  O.lnF  ^  =  AB^, 


O.lnF  ’ 
lk 

-nF 
t°  ^  , 

(2) 

Find  Max 
k 

,  •  ,  ~nF 
which  u^ 

(3) 

Calculate 

available 

(4) 

Calculate 

objective 

(5) 

Find  Max 

nF 

IKF 


K 

F 


lk 


and  set  it  equal 


and  fund  the  technology  objective,  k,  for 


(n+DF  _  nF 

Ukt  V 

DTlT 


A 

AV 


tttt: 


ikT  v*^ik’ 

objective  kf  already  funded  to  O.lnF^, 


for  each  technology 


-nF 
\  > 


A  HP 


0  .IF 


lk’ 


where  k  indicates  those  technology 


objectives  not  yet  funded,  add  to  the  funding  of  the 
technology  objective  that  satisfies  the  maximum  the  amount 
O.lF^t  or  O.lnF ^  as  appropriate,  set  the  latter  amount 
equal  to  AB2 , 

Calculate  B2  -  AB^  =  B^  and,  in  general, 

T3  AT3  —  "D 

I  “  I  1+1,  where  I  is  the  number  of  allocation  iterations, 
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Repeat  steps  (4),  (5),  and  (6)  until  one  of  the  following 

conditions  holds 

(a)  BI+1  =  °» 

(b)  each  remaining  possible  increment  of  funding,  O.lnF^ 
for  those  technology  objectives  not  yet  funded  or 
O.IF^t  for  those  technology  objectives  funded  to  some 
extent  (but  not  to  Fik->>  is  greater  than  BI+1, 

(c)  all  technology  objectives  k  are  funded  to  their 
respective  F^>  or 

(d)  all  technology  objectives  are  funded  to  some  extent, 

*  nF 
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